climateprediction.net home page
Task 22439837

Task 22439837

Name oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0
Workunit 12288175
Created 7 Jun 2024, 20:00:45 UTC
Sent 8 Jun 2024, 6:49:09 UTC
Report deadline 7 Aug 2024, 6:49:09 UTC
Received 12 Jun 2024, 11:21:19 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1488917
Run time 4 days 1 hours 33 min 57 sec
CPU time 17 hours 12 min 59 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,016.09 MB
Peak swap size 5,980.85 MB
Peak disk usage 1,899.80 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
1718036293:0.003] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:61bc11710756:1:1:96:96] [20240610:161813:1718036293:0.003] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:61bc11710756:1:1:96:96] [20240610:161813:1718036293:0.003] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:61bc11710756:1:1:96:96] [20240610:161813:1718036293:0.003] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:61bc11710756:1:1:96:96] [20240610:161813:1718036293:0.003] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:61bc11710756:1:1:96:96] [20240610:161813:1718036293:0.003] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:24:33 STEP  816 H= 204:00 +CPU= 60.453
  16:29:42 STEP  817 H= 204:15 +CPU= 56.654
  16:34:53 STEP  818 H= 204:30 +CPU= 57.864
  16:40:18 STEP  819 H= 204:45 +CPU= 61.194
  16:45:29 STEP  820 H= 205:00 +CPU= 58.000
  16:50:43 STEP  821 H= 205:15 +CPU= 58.084
  16:56:07 STEP  822 H= 205:30 +CPU= 59.411
  17:01:27 STEP  823 H= 205:45 +CPU= 58.197
  17:06:29 STEP  824 H= 206:00 +CPU= 54.803
  17:12:11 STEP  825 H= 206:15 +CPU= 61.262
  17:17:36 STEP  826 H= 206:30 +CPU= 59.055
  17:22:11 STEP  827 H= 206:45 +CPU= 50.774
  17:28:10 STEP  828 H= 207:00 +CPU= 64.631
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000828
  17:33:00 STEP  829 H= 207:15 +CPU= 52.231
  17:37:36 STEP  830 H= 207:30 +CPU= 49.939
  17:41:41 STEP  831 H= 207:45 +CPU= 44.052
  17:45:36 STEP  832 H= 208:00 +CPU= 43.398
  17:50:04 STEP  833 H= 208:15 +CPU= 47.775
  17:55:16 STEP  834 H= 208:30 +CPU= 56.329
  18:01:04 STEP  835 H= 208:45 +CPU= 63.430
  18:06:41 STEP  836 H= 209:00 +CPU= 61.687
  18:12:15 STEP  837 H= 209:15 +CPU= 60.464
  18:18:26 STEP  838 H= 209:30 +CPU= 65.188
  18:24:03 STEP  839 H= 209:45 +CPU= 60.411
  18:31:10 STEP  840 H= 210:00 +CPU= 74.164
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000840
  18:36:29 STEP  841 H= 210:15 +CPU= 56.087
  18:41:54 STEP  842 H= 210:30 +CPU= 57.743
  18:46:52 STEP  843 H= 210:45 +CPU= 51.969
  18:51:33 STEP  844 H= 211:00 +CPU= 48.002
  18:56:08 STEP  845 H= 211:15 +CPU= 48.602
  19:00:25 STEP  846 H= 211:30 +CPU= 45.926
  19:05:05 STEP  847 H= 211:45 +CPU= 48.608
  19:09:18 STEP  848 H= 212:00 +CPU= 44.248
  19:12:58 STEP  849 H= 212:15 +CPU= 38.592
  19:16:32 STEP  850 H= 212:30 +CPU= 37.649
  19:20:04 STEP  851 H= 212:45 +CPU= 36.959
  19:24:35 STEP  852 H= 213:00 +CPU= 46.981
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000852
  19:28:12 STEP  853 H= 213:15 +CPU= 37.451
  19:31:46 STEP  854 H= 213:30 +CPU= 37.458
  19:35:23 STEP  855 H= 213:45 +CPU= 37.979
  19:39:04 STEP  856 H= 214:00 +CPU= 38.510
  19:42:42 STEP  857 H= 214:15 +CPU= 37.809
  19:46:26 STEP  858 H= 214:30 +CPU= 38.605
  19:50:10 STEP  859 H= 214:45 +CPU= 38.763
  19:53:59 STEP  860 H= 215:00 +CPU= 39.497
  19:57:43 STEP  861 H= 215:15 +CPU= 38.497
  20:01:32 STEP  862 H= 215:30 +CPU= 38.935
  20:05:13 STEP  863 H= 215:45 +CPU= 38.245
  20:10:00 STEP  864 H= 216:00 +CPU= 49.972
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_8.zip
Uploading the intermediate file: upload_file_8.zip
  20:13:40 STEP  865 H= 216:15 +CPU= 38.056
  20:17:10 STEP  866 H= 216:30 +CPU= 36.943
  20:20:46 STEP  867 H= 216:45 +CPU= 37.373
  20:24:25 STEP  868 H= 217:00 +CPU= 38.295
  20:28:00 STEP  869 H= 217:15 +CPU= 37.403
  20:31:33 STEP  870 H= 217:30 +CPU= 37.376
  20:35:06 STEP  871 H= 217:45 +CPU= 37.189
  20:38:43 STEP  872 H= 218:00 +CPU= 37.745
  20:42:17 STEP  873 H= 218:15 +CPU= 37.565
  20:45:59 STEP  874 H= 218:30 +CPU= 38.352
  20:49:35 STEP  875 H= 218:45 +CPU= 37.718
  20:54:05 STEP  876 H= 219:00 +CPU= 47.076
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000876
  20:57:46 STEP  877 H= 219:15 +CPU= 38.241
  21:01:23 STEP  878 H= 219:30 +CPU= 38.113
  21:05:07 STEP  879 H= 219:45 +CPU= 38.364
  21:08:51 STEP  880 H= 220:00 +CPU= 38.872
  21:12:29 STEP  881 H= 220:15 +CPU= 37.837
  21:16:06 STEP  882 H= 220:30 +CPU= 37.912
  21:19:45 STEP  883 H= 220:45 +CPU= 37.923
  21:23:25 STEP  884 H= 221:00 +CPU= 38.207
  21:27:03 STEP  885 H= 221:15 +CPU= 37.729
  21:30:38 STEP  886 H= 221:30 +CPU= 37.365
  21:34:14 STEP  887 H= 221:45 +CPU= 37.557
  21:38:41 STEP  888 H= 222:00 +CPU= 46.637
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000888
  21:42:18 STEP  889 H= 222:15 +CPU= 37.923
  21:45:53 STEP  890 H= 222:30 +CPU= 37.991
  21:49:31 STEP  891 H= 222:45 +CPU= 38.620
  21:53:11 STEP  892 H= 223:00 +CPU= 38.652
  21:56:50 STEP  893 H= 223:15 +CPU= 38.679
  22:00:29 STEP  894 H= 223:30 +CPU= 38.602
  22:04:08 STEP  895 H= 223:45 +CPU= 38.589
  22:07:51 STEP  896 H= 224:00 +CPU= 39.020
  22:11:29 STEP  897 H= 224:15 +CPU= 38.132
  22:15:08 STEP  898 H= 224:30 +CPU= 37.867
  22:18:49 STEP  899 H= 224:45 +CPU= 38.113
  22:23:16 STEP  900 H= 225:00 +CPU= 46.251
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000900
  22:26:56 STEP  901 H= 225:15 +CPU= 38.088
  22:30:37 STEP  902 H= 225:30 +CPU= 37.988
  22:34:20 STEP  903 H= 225:45 +CPU= 38.350
  22:38:05 STEP  904 H= 226:00 +CPU= 39.041
  22:41:46 STEP  905 H= 226:15 +CPU= 38.230
  22:45:24 STEP  906 H= 226:30 +CPU= 38.310
  22:49:05 STEP  907 H= 226:45 +CPU= 38.652
  22:52:49 STEP  908 H= 227:00 +CPU= 38.770
  22:56:31 STEP  909 H= 227:15 +CPU= 38.500
  23:00:14 STEP  910 H= 227:30 +CPU= 38.852
  23:03:55 STEP  911 H= 227:45 +CPU= 38.593
  23:08:43 STEP  912 H= 228:00 +CPU= 50.652
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000912
  23:12:27 STEP  913 H= 228:15 +CPU= 38.874
  23:15:51 STEP  914 H= 228:30 +CPU= 37.586
  23:19:09 STEP  915 H= 228:45 +CPU= 37.225
  23:22:36 STEP  916 H= 229:00 +CPU= 38.260
  23:26:00 STEP  917 H= 229:15 +CPU= 37.605
  23:29:24 STEP  918 H= 229:30 +CPU= 37.689
  23:32:44 STEP  919 H= 229:45 +CPU= 37.348
  23:36:13 STEP  920 H= 230:00 +CPU= 38.726
  23:39:36 STEP  921 H= 230:15 +CPU= 37.325
  23:42:59 STEP  922 H= 230:30 +CPU= 37.276
  23:46:25 STEP  923 H= 230:45 +CPU= 37.540
  23:50:47 STEP  924 H= 231:00 +CPU= 48.377
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000924
  23:55:02 STEP  925 H= 231:15 +CPU= 44.332
  23:59:16 STEP  926 H= 231:30 +CPU= 44.642
  00:04:02 STEP  927 H= 231:45 +CPU= 49.691
  00:08:59 STEP  928 H= 232:00 +CPU= 51.860
  00:14:01 STEP  929 H= 232:15 +CPU= 53.156
  00:19:16 STEP  930 H= 232:30 +CPU= 55.166
  00:24:14 STEP  931 H= 232:45 +CPU= 51.464
  00:29:04 STEP  932 H= 233:00 +CPU= 50.253
  00:33:45 STEP  933 H= 233:15 +CPU= 49.139
  00:38:24 STEP  934 H= 233:30 +CPU= 49.275
  00:42:47 STEP  935 H= 233:45 +CPU= 46.784
  00:48:00 STEP  936 H= 234:00 +CPU= 55.675
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000936
  00:52:15 STEP  937 H= 234:15 +CPU= 45.909
  00:56:36 STEP  938 H= 234:30 +CPU= 47.020
  01:00:53 STEP  939 H= 234:45 +CPU= 46.209
  01:05:40 STEP  940 H= 235:00 +CPU= 50.487
  01:10:32 STEP  941 H= 235:15 +CPU= 51.732
  01:15:27 STEP  942 H= 235:30 +CPU= 52.793
  01:20:17 STEP  943 H= 235:45 +CPU= 50.926
  01:25:33 STEP  944 H= 236:00 +CPU= 54.739
  01:30:44 STEP  945 H= 236:15 +CPU= 53.760
  01:35:36 STEP  946 H= 236:30 +CPU= 50.207
  01:40:20 STEP  947 H= 236:45 +CPU= 50.188
  01:45:46 STEP  948 H= 237:00 +CPU= 57.421
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000948
  01:50:16 STEP  949 H= 237:15 +CPU= 46.718
  01:54:34 STEP  950 H= 237:30 +CPU= 44.914
  01:59:16 STEP  951 H= 237:45 +CPU= 48.007
  02:03:48 STEP  952 H= 238:00 +CPU= 46.453
  02:07:58 STEP  953 H= 238:15 +CPU= 43.203
  02:11:57 STEP  954 H= 238:30 +CPU= 41.793
  02:16:05 STEP  955 H= 238:45 +CPU= 43.046
  02:20:14 STEP  956 H= 239:00 +CPU= 42.631
  02:24:18 STEP  957 H= 239:15 +CPU= 42.003
  02:28:21 STEP  958 H= 239:30 +CPU= 41.932
  02:32:20 STEP  959 H= 239:45 +CPU= 41.110
  02:37:28 STEP  960 H= 240:00 +CPU= 52.956
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_9.zip
Uploading the intermediate file: upload_file_9.zip
  02:41:40 STEP  961 H= 240:15 +CPU= 42.234
  02:45:31 STEP  962 H= 240:30 +CPU= 39.852
  02:49:20 STEP  963 H= 240:45 +CPU= 39.333
  02:53:26 STEP  964 H= 241:00 +CPU= 42.117
  02:57:34 STEP  965 H= 241:15 +CPU= 41.869
  03:01:42 STEP  966 H= 241:30 +CPU= 41.901
  03:05:52 STEP  967 H= 241:45 +CPU= 42.336
  03:10:04 STEP  968 H= 242:00 +CPU= 42.950
  03:14:02 STEP  969 H= 242:15 +CPU= 40.840
  03:18:09 STEP  970 H= 242:30 +CPU= 41.708
  03:22:12 STEP  971 H= 242:45 +CPU= 40.902
  03:27:05 STEP  972 H= 243:00 +CPU= 49.569
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000972
  03:30:59 STEP  973 H= 243:15 +CPU= 39.646
  03:34:54 STEP  974 H= 243:30 +CPU= 40.378
  03:38:50 STEP  975 H= 243:45 +CPU= 40.443
  03:42:51 STEP  976 H= 244:00 +CPU= 41.024
  03:46:46 STEP  977 H= 244:15 +CPU= 39.842
  03:50:35 STEP  978 H= 244:30 +CPU= 39.365
  03:54:26 STEP  979 H= 244:45 +CPU= 39.422
  03:58:16 STEP  980 H= 245:00 +CPU= 39.990
  04:02:07 STEP  981 H= 245:15 +CPU= 39.089
  04:06:01 STEP  982 H= 245:30 +CPU= 39.836
  04:09:53 STEP  983 H= 245:45 +CPU= 40.211
  04:14:38 STEP  984 H= 246:00 +CPU= 48.641
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000984
  04:18:26 STEP  985 H= 246:15 +CPU= 39.122
  04:22:22 STEP  986 H= 246:30 +CPU= 39.962
  04:26:16 STEP  987 H= 246:45 +CPU= 39.476
  04:30:16 STEP  988 H= 247:00 +CPU= 40.200
  04:34:09 STEP  989 H= 247:15 +CPU= 40.140
  04:37:49 STEP  990 H= 247:30 +CPU= 37.993
  04:41:25 STEP  991 H= 247:45 +CPU= 37.609
  04:45:03 STEP  992 H= 248:00 +CPU= 38.076
  04:48:45 STEP  993 H= 248:15 +CPU= 37.967
  04:52:25 STEP  994 H= 248:30 +CPU= 37.819
  04:56:13 STEP  995 H= 248:45 +CPU= 38.501
  05:00:57 STEP  996 H= 249:00 +CPU= 48.631
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+000996
  05:04:46 STEP  997 H= 249:15 +CPU= 39.316
  05:08:35 STEP  998 H= 249:30 +CPU= 38.972
  05:12:26 STEP  999 H= 249:45 +CPU= 39.902
  05:16:22 STEP 1000 H= 250:00 +CPU= 40.601
  05:20:07 STEP 1001 H= 250:15 +CPU= 39.316
  05:24:02 STEP 1002 H= 250:30 +CPU= 40.276
  05:27:52 STEP 1003 H= 250:45 +CPU= 39.416
  05:31:49 STEP 1004 H= 251:00 +CPU= 40.374
  05:35:39 STEP 1005 H= 251:15 +CPU= 39.223
  05:39:18 STEP 1006 H= 251:30 +CPU= 38.060
  05:43:07 STEP 1007 H= 251:45 +CPU= 39.271
  05:47:48 STEP 1008 H= 252:00 +CPU= 50.830
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001008
  05:51:31 STEP 1009 H= 252:15 +CPU= 38.118
  05:55:08 STEP 1010 H= 252:30 +CPU= 37.912
  05:58:59 STEP 1011 H= 252:45 +CPU= 39.186
  06:02:53 STEP 1012 H= 253:00 +CPU= 40.475
  06:06:42 STEP 1013 H= 253:15 +CPU= 39.836
  06:10:41 STEP 1014 H= 253:30 +CPU= 40.268
  06:14:32 STEP 1015 H= 253:45 +CPU= 39.773
  06:18:30 STEP 1016 H= 254:00 +CPU= 40.864
  06:22:27 STEP 1017 H= 254:15 +CPU= 40.806
  06:26:20 STEP 1018 H= 254:30 +CPU= 39.931
  06:30:05 STEP 1019 H= 254:45 +CPU= 39.884
  06:34:48 STEP 1020 H= 255:00 +CPU= 48.885
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001020
  06:38:26 STEP 1021 H= 255:15 +CPU= 38.179
  06:42:19 STEP 1022 H= 255:30 +CPU= 40.081
  06:46:07 STEP 1023 H= 255:45 +CPU= 39.438
  06:49:51 STEP 1024 H= 256:00 +CPU= 39.397
  06:53:38 STEP 1025 H= 256:15 +CPU= 39.140
  06:57:30 STEP 1026 H= 256:30 +CPU= 39.256
  07:01:23 STEP 1027 H= 256:45 +CPU= 39.890
  07:05:07 STEP 1028 H= 257:00 +CPU= 39.796
  07:09:18 STEP 1029 H= 257:15 +CPU= 42.183
  07:13:09 STEP 1030 H= 257:30 +CPU= 39.972
  07:17:01 STEP 1031 H= 257:45 +CPU= 39.157
  07:21:39 STEP 1032 H= 258:00 +CPU= 48.657
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001032
  07:25:47 STEP 1033 H= 258:15 +CPU= 40.632
  07:29:37 STEP 1034 H= 258:30 +CPU= 39.849
  07:33:31 STEP 1035 H= 258:45 +CPU= 39.034
  07:37:30 STEP 1036 H= 259:00 +CPU= 39.968
  07:41:15 STEP 1037 H= 259:15 +CPU= 38.800
  07:45:03 STEP 1038 H= 259:30 +CPU= 38.444
  07:48:55 STEP 1039 H= 259:45 +CPU= 38.698
  07:52:50 STEP 1040 H= 260:00 +CPU= 40.221
  07:56:44 STEP 1041 H= 260:15 +CPU= 39.091
  08:00:30 STEP 1042 H= 260:30 +CPU= 38.060
  08:04:19 STEP 1043 H= 260:45 +CPU= 38.204
  08:09:10 STEP 1044 H= 261:00 +CPU= 48.551
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001044
  08:13:00 STEP 1045 H= 261:15 +CPU= 38.876
  08:16:55 STEP 1046 H= 261:30 +CPU= 39.730
  08:20:44 STEP 1047 H= 261:45 +CPU= 38.911
  08:24:40 STEP 1048 H= 262:00 +CPU= 39.464
  08:28:33 STEP 1049 H= 262:15 +CPU= 39.003
  08:32:14 STEP 1050 H= 262:30 +CPU= 38.234
  08:35:56 STEP 1051 H= 262:45 +CPU= 38.743
  08:39:50 STEP 1052 H= 263:00 +CPU= 39.063
  08:43:46 STEP 1053 H= 263:15 +CPU= 39.345
  08:47:39 STEP 1054 H= 263:30 +CPU= 38.921
  08:51:19 STEP 1055 H= 263:45 +CPU= 37.698
  08:56:18 STEP 1056 H= 264:00 +CPU= 51.406
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_10.zip
Uploading the intermediate file: upload_file_10.zip
  09:00:09 STEP 1057 H= 264:15 +CPU= 39.087
  09:04:08 STEP 1058 H= 264:30 +CPU= 39.172
  09:07:52 STEP 1059 H= 264:45 +CPU= 38.049
  09:11:42 STEP 1060 H= 265:00 +CPU= 38.615
  09:15:40 STEP 1061 H= 265:15 +CPU= 38.887
  09:19:38 STEP 1062 H= 265:30 +CPU= 38.972
  09:23:34 STEP 1063 H= 265:45 +CPU= 39.809
  09:27:38 STEP 1064 H= 266:00 +CPU= 39.838
  09:31:27 STEP 1065 H= 266:15 +CPU= 38.973
  09:35:08 STEP 1066 H= 266:30 +CPU= 37.631
  09:39:05 STEP 1067 H= 266:45 +CPU= 38.718
  09:43:56 STEP 1068 H= 267:00 +CPU= 48.864
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001068
  09:47:50 STEP 1069 H= 267:15 +CPU= 39.225
  09:51:52 STEP 1070 H= 267:30 +CPU= 40.188
  09:55:46 STEP 1071 H= 267:45 +CPU= 38.678
  09:59:40 STEP 1072 H= 268:00 +CPU= 39.490
  10:03:37 STEP 1073 H= 268:15 +CPU= 40.313
  10:07:30 STEP 1074 H= 268:30 +CPU= 39.280
  10:11:22 STEP 1075 H= 268:45 +CPU= 38.463
  10:15:19 STEP 1076 H= 269:00 +CPU= 39.850
  10:19:11 STEP 1077 H= 269:15 +CPU= 38.749
  10:22:56 STEP 1078 H= 269:30 +CPU= 38.910
  10:26:52 STEP 1079 H= 269:45 +CPU= 39.466
  10:31:57 STEP 1080 H= 270:00 +CPU= 50.474
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001080
  10:36:03 STEP 1081 H= 270:15 +CPU= 40.284
  10:40:15 STEP 1082 H= 270:30 +CPU= 40.351
  10:44:11 STEP 1083 H= 270:45 +CPU= 39.060
  10:48:09 STEP 1084 H= 271:00 +CPU= 40.381
  10:52:09 STEP 1085 H= 271:15 +CPU= 40.027
  10:56:03 STEP 1086 H= 271:30 +CPU= 38.645
  11:00:05 STEP 1087 H= 271:45 +CPU= 39.057
  11:04:06 STEP 1088 H= 272:00 +CPU= 39.205
  11:07:59 STEP 1089 H= 272:15 +CPU= 38.823
  11:11:57 STEP 1090 H= 272:30 +CPU= 39.522
  11:16:03 STEP 1091 H= 272:45 +CPU= 40.097
  11:20:58 STEP 1092 H= 273:00 +CPU= 48.762
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001092
  11:24:55 STEP 1093 H= 273:15 +CPU= 39.101
  11:28:59 STEP 1094 H= 273:30 +CPU= 39.118
  11:32:58 STEP 1095 H= 273:45 +CPU= 39.711
  11:37:09 STEP 1096 H= 274:00 +CPU= 40.908
  11:41:14 STEP 1097 H= 274:15 +CPU= 39.718
  11:45:15 STEP 1098 H= 274:30 +CPU= 39.490
  11:49:00 STEP 1099 H= 274:45 +CPU= 38.158
  11:52:58 STEP 1100 H= 275:00 +CPU= 38.851
  11:56:57 STEP 1101 H= 275:15 +CPU= 39.083
  12:00:58 STEP 1102 H= 275:30 +CPU= 39.478
  12:05:06 STEP 1103 H= 275:45 +CPU= 39.458
  12:10:20 STEP 1104 H= 276:00 +CPU= 51.683
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001104
  12:14:17 STEP 1105 H= 276:15 +CPU= 39.244
  12:18:11 STEP 1106 H= 276:30 +CPU= 40.442
  12:22:34 STEP 1107 H= 276:45 +CPU= 45.381
  12:27:35 STEP 1108 H= 277:00 +CPU= 52.256
  12:32:59 STEP 1109 H= 277:15 +CPU= 54.996
  12:38:14 STEP 1110 H= 277:30 +CPU= 53.478
  12:43:10 STEP 1111 H= 277:45 +CPU= 51.945
  12:48:18 STEP 1112 H= 278:00 +CPU= 53.099
  12:53:25 STEP 1113 H= 278:15 +CPU= 52.425
  12:58:25 STEP 1114 H= 278:30 +CPU= 51.099
  13:03:22 STEP 1115 H= 278:45 +CPU= 51.374
  13:08:54 STEP 1116 H= 279:00 +CPU= 58.662
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001116
  13:13:07 STEP 1117 H= 279:15 +CPU= 46.102
  13:17:14 STEP 1118 H= 279:30 +CPU= 44.428
  13:21:25 STEP 1119 H= 279:45 +CPU= 44.011
  13:25:40 STEP 1120 H= 280:00 +CPU= 45.254
  13:30:11 STEP 1121 H= 280:15 +CPU= 47.861
  13:34:46 STEP 1122 H= 280:30 +CPU= 48.816
  13:39:23 STEP 1123 H= 280:45 +CPU= 49.521
  13:44:06 STEP 1124 H= 281:00 +CPU= 50.107
  13:48:30 STEP 1125 H= 281:15 +CPU= 46.794
  13:52:49 STEP 1126 H= 281:30 +CPU= 46.663
  13:57:02 STEP 1127 H= 281:45 +CPU= 45.417
  14:02:09 STEP 1128 H= 282:00 +CPU= 54.837
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001128
  14:06:07 STEP 1129 H= 282:15 +CPU= 42.258
  14:10:03 STEP 1130 H= 282:30 +CPU= 41.579
  14:14:00 STEP 1131 H= 282:45 +CPU= 42.009
  14:18:10 STEP 1132 H= 283:00 +CPU= 43.956
  14:22:17 STEP 1133 H= 283:15 +CPU= 43.206
  14:26:14 STEP 1134 H= 283:30 +CPU= 41.885
  14:29:55 STEP 1135 H= 283:45 +CPU= 38.987
  14:34:01 STEP 1136 H= 284:00 +CPU= 41.164
  14:38:07 STEP 1137 H= 284:15 +CPU= 40.857
  14:42:05 STEP 1138 H= 284:30 +CPU= 40.349
  14:45:54 STEP 1139 H= 284:45 +CPU= 39.516
  14:50:38 STEP 1140 H= 285:00 +CPU= 48.706
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001140
  14:54:26 STEP 1141 H= 285:15 +CPU= 39.086
  14:58:29 STEP 1142 H= 285:30 +CPU= 39.910
  15:02:15 STEP 1143 H= 285:45 +CPU= 38.736
  15:06:14 STEP 1144 H= 286:00 +CPU= 41.126
  15:10:00 STEP 1145 H= 286:15 +CPU= 39.338
  15:13:55 STEP 1146 H= 286:30 +CPU= 39.383
  15:17:45 STEP 1147 H= 286:45 +CPU= 40.191
  15:21:32 STEP 1148 H= 287:00 +CPU= 39.688
  15:25:24 STEP 1149 H= 287:15 +CPU= 40.522
  15:29:18 STEP 1150 H= 287:30 +CPU= 40.070
  15:33:17 STEP 1151 H= 287:45 +CPU= 40.388
  15:38:10 STEP 1152 H= 288:00 +CPU= 51.571
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_11.zip
Uploading the intermediate file: upload_file_11.zip
  15:42:01 STEP 1153 H= 288:15 +CPU= 39.915
  15:45:46 STEP 1154 H= 288:30 +CPU= 39.403
  15:49:32 STEP 1155 H= 288:45 +CPU= 39.041
  15:53:19 STEP 1156 H= 289:00 +CPU= 40.150
  15:57:09 STEP 1157 H= 289:15 +CPU= 39.320
  16:00:50 STEP 1158 H= 289:30 +CPU= 38.290
  16:04:36 STEP 1159 H= 289:45 +CPU= 39.726
  16:08:30 STEP 1160 H= 290:00 +CPU= 40.373
  16:12:06 STEP 1161 H= 290:15 +CPU= 38.341
  16:15:51 STEP 1162 H= 290:30 +CPU= 39.372
  16:19:27 STEP 1163 H= 290:45 +CPU= 38.458
  16:24:00 STEP 1164 H= 291:00 +CPU= 47.317
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001164
  16:27:44 STEP 1165 H= 291:15 +CPU= 39.463
  16:31:37 STEP 1166 H= 291:30 +CPU= 39.898
  16:35:30 STEP 1167 H= 291:45 +CPU= 40.003
  16:39:16 STEP 1168 H= 292:00 +CPU= 40.072
  16:42:55 STEP 1169 H= 292:15 +CPU= 38.926
  16:46:37 STEP 1170 H= 292:30 +CPU= 38.504
  16:50:20 STEP 1171 H= 292:45 +CPU= 39.490
  16:54:14 STEP 1172 H= 293:00 +CPU= 40.453
  16:58:06 STEP 1173 H= 293:15 +CPU= 39.538
  17:01:50 STEP 1174 H= 293:30 +CPU= 39.227
  17:05:32 STEP 1175 H= 293:45 +CPU= 38.918
  17:09:56 STEP 1176 H= 294:00 +CPU= 47.537
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001176
  17:13:34 STEP 1177 H= 294:15 +CPU= 39.002
  17:17:14 STEP 1178 H= 294:30 +CPU= 38.920
  17:21:01 STEP 1179 H= 294:45 +CPU= 39.550
  17:24:51 STEP 1180 H= 295:00 +CPU= 40.237
  17:28:28 STEP 1181 H= 295:15 +CPU= 38.141
  17:32:12 STEP 1182 H= 295:30 +CPU= 38.344
  17:35:57 STEP 1183 H= 295:45 +CPU= 39.066
  17:39:43 STEP 1184 H= 296:00 +CPU= 39.452
  17:43:23 STEP 1185 H= 296:15 +CPU= 39.151
  17:46:50 STEP 1186 H= 296:30 +CPU= 37.231
  17:50:28 STEP 1187 H= 296:45 +CPU= 38.572
  17:54:47 STEP 1188 H= 297:00 +CPU= 46.389
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001188
  17:58:21 STEP 1189 H= 297:15 +CPU= 38.397
  18:02:04 STEP 1190 H= 297:30 +CPU= 38.755
  18:05:40 STEP 1191 H= 297:45 +CPU= 38.639
  18:09:25 STEP 1192 H= 298:00 +CPU= 39.776
  18:13:04 STEP 1193 H= 298:15 +CPU= 37.863
  18:16:38 STEP 1194 H= 298:30 +CPU= 37.802
  18:20:12 STEP 1195 H= 298:45 +CPU= 38.206
  18:23:47 STEP 1196 H= 299:00 +CPU= 38.704
  18:27:21 STEP 1197 H= 299:15 +CPU= 38.187
  18:31:14 STEP 1198 H= 299:30 +CPU= 39.508
  18:34:54 STEP 1199 H= 299:45 +CPU= 38.180
  18:39:48 STEP 1200 H= 300:00 +CPU= 50.444
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001200
  18:43:23 STEP 1201 H= 300:15 +CPU= 38.114
  18:47:02 STEP 1202 H= 300:30 +CPU= 38.010
  18:50:47 STEP 1203 H= 300:45 +CPU= 38.766
  18:54:32 STEP 1204 H= 301:00 +CPU= 38.901
  18:58:09 STEP 1205 H= 301:15 +CPU= 37.968
  19:01:49 STEP 1206 H= 301:30 +CPU= 37.920
  19:05:41 STEP 1207 H= 301:45 +CPU= 39.787
  19:09:29 STEP 1208 H= 302:00 +CPU= 39.845
  19:13:25 STEP 1209 H= 302:15 +CPU= 40.289
  19:17:08 STEP 1210 H= 302:30 +CPU= 38.775
  19:21:01 STEP 1211 H= 302:45 +CPU= 39.450
  19:25:33 STEP 1212 H= 303:00 +CPU= 47.150
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001212
  19:29:29 STEP 1213 H= 303:15 +CPU= 39.920
  19:33:10 STEP 1214 H= 303:30 +CPU= 38.760
  19:37:09 STEP 1215 H= 303:45 +CPU= 41.145
  19:41:06 STEP 1216 H= 304:00 +CPU= 40.494
  19:45:03 STEP 1217 H= 304:15 +CPU= 40.396
  19:48:50 STEP 1218 H= 304:30 +CPU= 38.489
  19:52:26 STEP 1219 H= 304:45 +CPU= 38.568
  19:56:12 STEP 1220 H= 305:00 +CPU= 39.036
  19:59:51 STEP 1221 H= 305:15 +CPU= 38.874
  20:03:29 STEP 1222 H= 305:30 +CPU= 38.555
  20:07:07 STEP 1223 H= 305:45 +CPU= 38.180
  20:11:28 STEP 1224 H= 306:00 +CPU= 46.113
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001224
  20:15:04 STEP 1225 H= 306:15 +CPU= 38.069
  20:18:38 STEP 1226 H= 306:30 +CPU= 37.557
  20:22:17 STEP 1227 H= 306:45 +CPU= 38.746
  20:25:43 STEP 1228 H= 307:00 +CPU= 37.500
  20:29:14 STEP 1229 H= 307:15 +CPU= 37.796
  20:32:39 STEP 1230 H= 307:30 +CPU= 36.956
  20:36:13 STEP 1231 H= 307:45 +CPU= 37.798
  20:39:56 STEP 1232 H= 308:00 +CPU= 39.418
  20:43:36 STEP 1233 H= 308:15 +CPU= 38.521
  20:47:30 STEP 1234 H= 308:30 +CPU= 39.573
  20:51:06 STEP 1235 H= 308:45 +CPU= 38.781
  20:55:36 STEP 1236 H= 309:00 +CPU= 47.426
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001236
  20:59:12 STEP 1237 H= 309:15 +CPU= 38.011
  21:02:54 STEP 1238 H= 309:30 +CPU= 38.713
  21:06:40 STEP 1239 H= 309:45 +CPU= 38.801
  21:10:32 STEP 1240 H= 310:00 +CPU= 39.614
  21:14:11 STEP 1241 H= 310:15 +CPU= 38.212
  21:17:53 STEP 1242 H= 310:30 +CPU= 37.969
  21:21:34 STEP 1243 H= 310:45 +CPU= 37.935
  21:25:22 STEP 1244 H= 311:00 +CPU= 38.947
  21:29:12 STEP 1245 H= 311:15 +CPU= 39.022
  21:32:50 STEP 1246 H= 311:30 +CPU= 37.922
  21:36:36 STEP 1247 H= 311:45 +CPU= 38.718
  21:41:31 STEP 1248 H= 312:00 +CPU= 50.432
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_12.zip
Uploading the intermediate file: upload_file_12.zip
  21:45:17 STEP 1249 H= 312:15 +CPU= 38.740
  21:49:05 STEP 1250 H= 312:30 +CPU= 38.950
  21:52:56 STEP 1251 H= 312:45 +CPU= 39.326
  21:56:36 STEP 1252 H= 313:00 +CPU= 38.642
  22:00:22 STEP 1253 H= 313:15 +CPU= 38.428
  22:03:59 STEP 1254 H= 313:30 +CPU= 37.931
  22:07:43 STEP 1255 H= 313:45 +CPU= 38.103
  22:11:29 STEP 1256 H= 314:00 +CPU= 39.156
  22:15:16 STEP 1257 H= 314:15 +CPU= 39.340
  22:18:56 STEP 1258 H= 314:30 +CPU= 38.287
  22:22:44 STEP 1259 H= 314:45 +CPU= 38.734
  22:27:10 STEP 1260 H= 315:00 +CPU= 46.138
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001260
  22:30:53 STEP 1261 H= 315:15 +CPU= 38.384
  22:34:37 STEP 1262 H= 315:30 +CPU= 38.223
  22:38:15 STEP 1263 H= 315:45 +CPU= 38.451
  22:41:56 STEP 1264 H= 316:00 +CPU= 39.260
  22:45:37 STEP 1265 H= 316:15 +CPU= 38.170
  22:49:20 STEP 1266 H= 316:30 +CPU= 38.465
  22:53:02 STEP 1267 H= 316:45 +CPU= 38.130
  22:56:50 STEP 1268 H= 317:00 +CPU= 39.186
  23:00:38 STEP 1269 H= 317:15 +CPU= 39.187
  23:04:27 STEP 1270 H= 317:30 +CPU= 39.071
  23:08:12 STEP 1271 H= 317:45 +CPU= 38.432
  23:12:36 STEP 1272 H= 318:00 +CPU= 46.314
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001272
  23:16:23 STEP 1273 H= 318:15 +CPU= 38.874
  23:20:06 STEP 1274 H= 318:30 +CPU= 38.217
  23:23:48 STEP 1275 H= 318:45 +CPU= 38.782
  23:27:35 STEP 1276 H= 319:00 +CPU= 39.190
  23:31:23 STEP 1277 H= 319:15 +CPU= 38.768
  23:35:11 STEP 1278 H= 319:30 +CPU= 39.119
  23:38:57 STEP 1279 H= 319:45 +CPU= 39.078
  23:42:58 STEP 1280 H= 320:00 +CPU= 40.685
  23:46:51 STEP 1281 H= 320:15 +CPU= 39.393
  23:50:40 STEP 1282 H= 320:30 +CPU= 39.166
  23:54:21 STEP 1283 H= 320:45 +CPU= 38.455
  23:58:51 STEP 1284 H= 321:00 +CPU= 46.782
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001284
  00:02:41 STEP 1285 H= 321:15 +CPU= 38.996
  00:06:27 STEP 1286 H= 321:30 +CPU= 38.785
  00:10:13 STEP 1287 H= 321:45 +CPU= 38.728
  00:14:09 STEP 1288 H= 322:00 +CPU= 39.875
  00:17:58 STEP 1289 H= 322:15 +CPU= 38.612
  00:21:36 STEP 1290 H= 322:30 +CPU= 37.824
  00:25:24 STEP 1291 H= 322:45 +CPU= 38.581
  00:29:11 STEP 1292 H= 323:00 +CPU= 39.814
  00:32:59 STEP 1293 H= 323:15 +CPU= 38.593
  00:36:52 STEP 1294 H= 323:30 +CPU= 39.344
  00:40:37 STEP 1295 H= 323:45 +CPU= 38.151
  00:45:35 STEP 1296 H= 324:00 +CPU= 51.067
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001296
  00:49:24 STEP 1297 H= 324:15 +CPU= 38.640
  00:53:09 STEP 1298 H= 324:30 +CPU= 38.474
  00:57:01 STEP 1299 H= 324:45 +CPU= 39.404
  01:00:56 STEP 1300 H= 325:00 +CPU= 39.416
  01:04:43 STEP 1301 H= 325:15 +CPU= 38.640
  01:08:40 STEP 1302 H= 325:30 +CPU= 39.729
  01:12:36 STEP 1303 H= 325:45 +CPU= 39.508
  01:16:31 STEP 1304 H= 326:00 +CPU= 39.832
  01:20:19 STEP 1305 H= 326:15 +CPU= 39.212
  01:24:13 STEP 1306 H= 326:30 +CPU= 39.013
  01:27:57 STEP 1307 H= 326:45 +CPU= 38.610
  01:32:41 STEP 1308 H= 327:00 +CPU= 47.913
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001308
  01:36:32 STEP 1309 H= 327:15 +CPU= 38.946
  01:40:19 STEP 1310 H= 327:30 +CPU= 38.960
  01:44:05 STEP 1311 H= 327:45 +CPU= 38.748
  01:47:56 STEP 1312 H= 328:00 +CPU= 39.378
  01:51:44 STEP 1313 H= 328:15 +CPU= 38.943
  01:55:38 STEP 1314 H= 328:30 +CPU= 39.799
  01:59:33 STEP 1315 H= 328:45 +CPU= 40.178
  02:03:30 STEP 1316 H= 329:00 +CPU= 40.585
  02:07:29 STEP 1317 H= 329:15 +CPU= 40.098
  02:11:25 STEP 1318 H= 329:30 +CPU= 39.535
  02:15:21 STEP 1319 H= 329:45 +CPU= 39.507
  02:19:57 STEP 1320 H= 330:00 +CPU= 47.180
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001320
  02:23:37 STEP 1321 H= 330:15 +CPU= 38.626
  02:27:25 STEP 1322 H= 330:30 +CPU= 38.762
  02:31:15 STEP 1323 H= 330:45 +CPU= 39.009
  02:35:08 STEP 1324 H= 331:00 +CPU= 39.450
  02:38:58 STEP 1325 H= 331:15 +CPU= 39.020
  02:42:46 STEP 1326 H= 331:30 +CPU= 39.024
  02:46:42 STEP 1327 H= 331:45 +CPU= 40.217
  02:50:36 STEP 1328 H= 332:00 +CPU= 39.844
  02:54:26 STEP 1329 H= 332:15 +CPU= 39.144
  02:58:19 STEP 1330 H= 332:30 +CPU= 39.474
  03:02:14 STEP 1331 H= 332:45 +CPU= 40.414
  03:07:01 STEP 1332 H= 333:00 +CPU= 49.079
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001332
  03:10:54 STEP 1333 H= 333:15 +CPU= 39.893
  03:14:44 STEP 1334 H= 333:30 +CPU= 39.872
  03:18:37 STEP 1335 H= 333:45 +CPU= 39.890
  03:22:27 STEP 1336 H= 334:00 +CPU= 40.098
  03:26:15 STEP 1337 H= 334:15 +CPU= 39.168
  03:29:59 STEP 1338 H= 334:30 +CPU= 38.859
  03:33:52 STEP 1339 H= 334:45 +CPU= 39.691
  03:37:53 STEP 1340 H= 335:00 +CPU= 41.327
  03:41:43 STEP 1341 H= 335:15 +CPU= 40.118
  03:45:32 STEP 1342 H= 335:30 +CPU= 39.710
  03:49:18 STEP 1343 H= 335:45 +CPU= 38.989
  03:54:27 STEP 1344 H= 336:00 +CPU= 52.978
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_13.zip
Uploading the intermediate file: upload_file_13.zip
  03:58:30 STEP 1345 H= 336:15 +CPU= 40.679
  04:02:37 STEP 1346 H= 336:30 +CPU= 40.526
  04:06:29 STEP 1347 H= 336:45 +CPU= 39.243
  04:10:32 STEP 1348 H= 337:00 +CPU= 40.232
  04:14:16 STEP 1349 H= 337:15 +CPU= 39.139
  04:17:56 STEP 1350 H= 337:30 +CPU= 38.468
  04:21:43 STEP 1351 H= 337:45 +CPU= 39.018
  04:25:33 STEP 1352 H= 338:00 +CPU= 39.351
  04:29:22 STEP 1353 H= 338:15 +CPU= 38.928
  04:33:10 STEP 1354 H= 338:30 +CPU= 38.919
  04:37:01 STEP 1355 H= 338:45 +CPU= 40.228
  04:41:39 STEP 1356 H= 339:00 +CPU= 48.312
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001356
  04:45:43 STEP 1357 H= 339:15 +CPU= 41.082
  04:49:43 STEP 1358 H= 339:30 +CPU= 40.040
  04:53:42 STEP 1359 H= 339:45 +CPU= 39.469
  04:57:58 STEP 1360 H= 340:00 +CPU= 41.519
  05:01:50 STEP 1361 H= 340:15 +CPU= 39.651
  05:05:44 STEP 1362 H= 340:30 +CPU= 40.183
  05:09:42 STEP 1363 H= 340:45 +CPU= 39.791
  05:13:58 STEP 1364 H= 341:00 +CPU= 41.439
  05:17:48 STEP 1365 H= 341:15 +CPU= 38.804
  05:21:50 STEP 1366 H= 341:30 +CPU= 40.260
  05:25:42 STEP 1367 H= 341:45 +CPU= 39.116
  05:30:28 STEP 1368 H= 342:00 +CPU= 48.591
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001368
  05:34:23 STEP 1369 H= 342:15 +CPU= 39.544
  05:38:20 STEP 1370 H= 342:30 +CPU= 40.200
  05:42:15 STEP 1371 H= 342:45 +CPU= 39.293
  05:46:24 STEP 1372 H= 343:00 +CPU= 40.293
  05:51:02 STEP 1373 H= 343:15 +CPU= 45.596
  05:55:41 STEP 1374 H= 343:30 +CPU= 47.993
  06:01:16 STEP 1375 H= 343:45 +CPU= 57.419
  06:07:39 STEP 1376 H= 344:00 +CPU= 66.200
  06:13:23 STEP 1377 H= 344:15 +CPU= 60.148
  06:19:07 STEP 1378 H= 344:30 +CPU= 60.440
  06:25:02 STEP 1379 H= 344:45 +CPU= 61.227
  06:32:19 STEP 1380 H= 345:00 +CPU= 76.111
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001380
  06:38:15 STEP 1381 H= 345:15 +CPU= 62.763
  06:43:51 STEP 1382 H= 345:30 +CPU= 59.133
  06:49:25 STEP 1383 H= 345:45 +CPU= 57.121
  06:55:14 STEP 1384 H= 346:00 +CPU= 60.125
  07:00:50 STEP 1385 H= 346:15 +CPU= 57.030
  07:05:51 STEP 1386 H= 346:30 +CPU= 51.077
  07:10:47 STEP 1387 H= 346:45 +CPU= 51.735
  07:16:06 STEP 1388 H= 347:00 +CPU= 54.097
  07:21:21 STEP 1389 H= 347:15 +CPU= 53.780
  07:26:10 STEP 1390 H= 347:30 +CPU= 49.315
  07:31:01 STEP 1391 H= 347:45 +CPU= 50.342
  07:37:48 STEP 1392 H= 348:00 +CPU= 70.920
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001392
  07:43:07 STEP 1393 H= 348:15 +CPU= 55.543
  07:48:36 STEP 1394 H= 348:30 +CPU= 56.398
  07:54:11 STEP 1395 H= 348:45 +CPU= 57.893
  07:59:41 STEP 1396 H= 349:00 +CPU= 57.272
  08:05:18 STEP 1397 H= 349:15 +CPU= 58.408
  08:10:38 STEP 1398 H= 349:30 +CPU= 56.100
  08:16:00 STEP 1399 H= 349:45 +CPU= 58.213
  08:21:11 STEP 1400 H= 350:00 +CPU= 55.374
  08:25:54 STEP 1401 H= 350:15 +CPU= 50.194
  08:30:27 STEP 1402 H= 350:30 +CPU= 48.635
  08:35:08 STEP 1403 H= 350:45 +CPU= 48.734
  08:41:10 STEP 1404 H= 351:00 +CPU= 61.646
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001404
  08:46:05 STEP 1405 H= 351:15 +CPU= 48.985
  08:50:14 STEP 1406 H= 351:30 +CPU= 43.251
  08:54:16 STEP 1407 H= 351:45 +CPU= 41.745
  08:58:15 STEP 1408 H= 352:00 +CPU= 41.571
  09:02:11 STEP 1409 H= 352:15 +CPU= 40.876
  09:06:14 STEP 1410 H= 352:30 +CPU= 41.686
  09:10:05 STEP 1411 H= 352:45 +CPU= 40.236
  09:14:05 STEP 1412 H= 353:00 +CPU= 41.905
  09:18:02 STEP 1413 H= 353:15 +CPU= 40.790
  09:22:46 STEP 1414 H= 353:30 +CPU= 46.551
  09:27:10 STEP 1415 H= 353:45 +CPU= 44.929
  09:32:57 STEP 1416 H= 354:00 +CPU= 58.998
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001416
  09:37:34 STEP 1417 H= 354:15 +CPU= 47.628
  09:42:12 STEP 1418 H= 354:30 +CPU= 47.262
  09:46:36 STEP 1419 H= 354:45 +CPU= 44.927
  09:51:29 STEP 1420 H= 355:00 +CPU= 48.689
  09:56:33 STEP 1421 H= 355:15 +CPU= 50.304
  10:01:11 STEP 1422 H= 355:30 +CPU= 46.212
  10:05:43 STEP 1423 H= 355:45 +CPU= 44.765
  10:09:56 STEP 1424 H= 356:00 +CPU= 42.620
  10:13:59 STEP 1425 H= 356:15 +CPU= 40.804
  10:17:53 STEP 1426 H= 356:30 +CPU= 39.565
  10:21:32 STEP 1427 H= 356:45 +CPU= 37.977
  10:26:00 STEP 1428 H= 357:00 +CPU= 46.984
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001428
  10:29:46 STEP 1429 H= 357:15 +CPU= 39.049
  10:33:22 STEP 1430 H= 357:30 +CPU= 37.518
  10:37:01 STEP 1431 H= 357:45 +CPU= 38.183
  10:40:48 STEP 1432 H= 358:00 +CPU= 39.362
  10:44:37 STEP 1433 H= 358:15 +CPU= 39.608
  10:48:30 STEP 1434 H= 358:30 +CPU= 40.601
  10:52:21 STEP 1435 H= 358:45 +CPU= 40.608
  10:56:14 STEP 1436 H= 359:00 +CPU= 41.545
  11:00:04 STEP 1437 H= 359:15 +CPU= 40.721
  11:03:50 STEP 1438 H= 359:30 +CPU= 40.324
  11:07:33 STEP 1439 H= 359:45 +CPU= 39.961
  11:12:16 STEP 1440 H= 360:00 +CPU= 49.144
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.7294870493E-21      I      -0.6100701445E-05      I
ICloud fractionI      0.0000000000E+00      I       0.3681712559E-01      I
I  Relat. Hum. I      0.4036084202E+00      I       0.4121103305E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.7849374259E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    317475620 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0132( 2.3%)
 11:12:16 STEP 1440 H= 360:00 +CPU= 49.144
 END CNT3
 NSTEP =  1440 CNT0   000000000  37.094  37.094 214.096   0.173               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        18590/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    328    0       46      1044068    18590 ************   195.3    18.6   176.7    24.8   461.7
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  677   34871.7   34871.7     15.29      0.00
   9 SPCM     - SPECTRAL COMP.                     624    6612.4    6612.4      2.67      0.00
  10 SCAN2M - PHYSICS                              625  125888.8  125888.8     50.94      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 52    4769.2    4769.2      0.16      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 27     363.9     363.9      0.01      0.00
  17 GRIDFPOS IN CNT4                               52     207.4     207.4      0.01      0.00
  27 RESTART FILES - WRITING                        13   19076.8   19076.8      0.16      0.00
  28 RESTART FILES - READING                         1   34287.4   34287.4      0.02      0.00
  29 SU4FPOS IN CNT4                                52       0.2       0.2      0.00      0.00
  30 DYNFPOS IN CNT4                                52   48822.1   48822.1      1.64      0.00
  31 POSDDH IN STEPO                                 3       3.2       3.2      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    625   46103.2   46103.2     18.66      0.00
  39 SU0YOMB                                         1    2908.9    2908.9      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    625     274.0     274.0      0.11      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              624    3130.5    3130.5      1.26      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              624     623.6     623.6      0.25      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 27       0.4       0.4      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1356    4369.2    4369.2      3.84      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1300    2302.7    2302.7      1.94      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1300     805.8     805.8      0.68      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1356    1309.0    1309.0      1.15      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1     200.7     200.7      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1356     414.3     414.3      0.36      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1300     215.1     215.1      0.18      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1356     730.0     730.0      0.64      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1300     483.4     483.4      0.41      0.00
 400 GSTATS                                      73253       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 68103       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME   154443.5 CPU TIME   26682.8 VECTOR TIME    26682.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY        8.4
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/3/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/3/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12288175/ICMUAh7zg+001356
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
11:17:12 (37): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0_r2025886852_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
12 Jun 2024 11:22:21 1488917 22439837 oifs_43r3_bl_a4wc_2016092300_20_1017_12288175_0 1,295,100 61,979 0.0479


©2024 cpdn.org