climateprediction.net home page
Task 22314745

Task 22314745

Name oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1
Workunit 12213102
Created 23 Feb 2023, 21:59:13 UTC
Sent 23 Feb 2023, 22:08:51 UTC
Report deadline 24 Apr 2023, 22:08:51 UTC
Received 28 Feb 2023, 9:43:30 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1528108
Run time 22 hours 9 min 3 sec
CPU time 20 hours 57 min 2 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 5.65 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,068.46 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,362.29 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<stderr_txt>
me/karl/slots/2/ICMSHh7zg+000804
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000804
  15:07:44 STEP  805 H= 201:15 +CPU= 51.780
  15:08:39 STEP  806 H= 201:30 +CPU= 53.346
  15:09:34 STEP  807 H= 201:45 +CPU= 53.424
  15:10:27 STEP  808 H= 202:00 +CPU= 51.680
  15:11:21 STEP  809 H= 202:15 +CPU= 52.895
  15:12:14 STEP  810 H= 202:30 +CPU= 51.493
  15:13:10 STEP  811 H= 202:45 +CPU= 54.077
  15:14:04 STEP  812 H= 203:00 +CPU= 52.988
  15:15:01 STEP  813 H= 203:15 +CPU= 54.883
  15:15:52 STEP  814 H= 203:30 +CPU= 49.977
  15:16:50 STEP  815 H= 203:45 +CPU= 56.156
  15:18:01 STEP  816 H= 204:00 +CPU= 68.523
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000816
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000816
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000816
  15:18:56 STEP  817 H= 204:15 +CPU= 53.061
  15:19:47 STEP  818 H= 204:30 +CPU= 49.608
  15:20:46 STEP  819 H= 204:45 +CPU= 56.858
  15:21:38 STEP  820 H= 205:00 +CPU= 51.000
  15:22:36 STEP  821 H= 205:15 +CPU= 55.774
  15:23:29 STEP  822 H= 205:30 +CPU= 52.024
  15:24:22 STEP  823 H= 205:45 +CPU= 51.082
  15:25:17 STEP  824 H= 206:00 +CPU= 53.539
  15:26:10 STEP  825 H= 206:15 +CPU= 51.585
  15:27:04 STEP  826 H= 206:30 +CPU= 53.039
  15:27:55 STEP  827 H= 206:45 +CPU= 50.117
  15:29:02 STEP  828 H= 207:00 +CPU= 65.005
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000828
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000828
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000828
  15:29:57 STEP  829 H= 207:15 +CPU= 53.750
  15:30:52 STEP  830 H= 207:30 +CPU= 52.909
  15:31:49 STEP  831 H= 207:45 +CPU= 55.128
  15:32:43 STEP  832 H= 208:00 +CPU= 52.285
  15:33:40 STEP  833 H= 208:15 +CPU= 55.392
  15:34:40 STEP  834 H= 208:30 +CPU= 58.987
  15:35:50 STEP  835 H= 208:45 +CPU= 67.595
  15:36:50 STEP  836 H= 209:00 +CPU= 58.883
  15:37:57 STEP  837 H= 209:15 +CPU= 64.275
  15:38:56 STEP  838 H= 209:30 +CPU= 57.749
  15:39:52 STEP  839 H= 209:45 +CPU= 54.971
  15:40:59 STEP  840 H= 210:00 +CPU= 65.211
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000840
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000840
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000840
  15:41:58 STEP  841 H= 210:15 +CPU= 57.084
  15:42:53 STEP  842 H= 210:30 +CPU= 53.541
  15:43:59 STEP  843 H= 210:45 +CPU= 64.222
  15:45:00 STEP  844 H= 211:00 +CPU= 59.565
  15:45:57 STEP  845 H= 211:15 +CPU= 54.961
  15:46:52 STEP  846 H= 211:30 +CPU= 53.565
  15:47:50 STEP  847 H= 211:45 +CPU= 55.860
  15:48:45 STEP  848 H= 212:00 +CPU= 54.276
  15:49:43 STEP  849 H= 212:15 +CPU= 55.580
  15:50:46 STEP  850 H= 212:30 +CPU= 61.390
  15:51:45 STEP  851 H= 212:45 +CPU= 57.243
  15:53:03 STEP  852 H= 213:00 +CPU= 75.592
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000852
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000852
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000852
  15:54:05 STEP  853 H= 213:15 +CPU= 60.894
  15:55:05 STEP  854 H= 213:30 +CPU= 58.645
  15:55:59 STEP  855 H= 213:45 +CPU= 51.885
  15:56:56 STEP  856 H= 214:00 +CPU= 54.732
  15:57:48 STEP  857 H= 214:15 +CPU= 50.606
  15:58:52 STEP  858 H= 214:30 +CPU= 61.706
  15:59:48 STEP  859 H= 214:45 +CPU= 55.298
  16:00:56 STEP  860 H= 215:00 +CPU= 65.820
  16:01:55 STEP  861 H= 215:15 +CPU= 57.556
  16:03:00 STEP  862 H= 215:30 +CPU= 62.681
  16:03:57 STEP  863 H= 215:45 +CPU= 55.027
  16:05:14 STEP  864 H= 216:00 +CPU= 74.176
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000864
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000864
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000864
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000768
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000768
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000768
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000780
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000780
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000780
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000792
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000792
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000792
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000804
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000804
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000804
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000816
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000816
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000816
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000828
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000828
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000828
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000840
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000840
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000840
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000852
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000852
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000852
Zipping up the intermediate file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_8.zip
Uploading the intermediate file: upload_file_8.zip
  16:06:06 STEP  865 H= 216:15 +CPU= 47.908
  16:07:04 STEP  866 H= 216:30 +CPU= 55.935
  16:08:01 STEP  867 H= 216:45 +CPU= 55.797
  16:09:09 STEP  868 H= 217:00 +CPU= 65.526
  16:10:09 STEP  869 H= 217:15 +CPU= 58.911
  16:11:13 STEP  870 H= 217:30 +CPU= 62.021
  16:12:16 STEP  871 H= 217:45 +CPU= 60.987
  16:13:16 STEP  872 H= 218:00 +CPU= 58.501
  16:14:10 STEP  873 H= 218:15 +CPU= 52.111
  16:15:02 STEP  874 H= 218:30 +CPU= 50.805
  16:15:58 STEP  875 H= 218:45 +CPU= 54.183
  16:17:11 STEP  876 H= 219:00 +CPU= 71.279
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000876
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000876
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000876
  16:18:14 STEP  877 H= 219:15 +CPU= 61.646
  16:19:17 STEP  878 H= 219:30 +CPU= 61.173
  16:20:19 STEP  879 H= 219:45 +CPU= 60.472
  16:21:19 STEP  880 H= 220:00 +CPU= 58.069
  16:22:18 STEP  881 H= 220:15 +CPU= 57.806
  16:23:13 STEP  882 H= 220:30 +CPU= 53.275
  16:24:10 STEP  883 H= 220:45 +CPU= 55.795
  16:25:03 STEP  884 H= 221:00 +CPU= 51.375
  16:26:01 STEP  885 H= 221:15 +CPU= 56.874
  16:27:04 STEP  886 H= 221:30 +CPU= 60.657
  16:28:08 STEP  887 H= 221:45 +CPU= 62.333
  16:29:27 STEP  888 H= 222:00 +CPU= 77.439
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000888
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000888
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000888
  16:30:31 STEP  889 H= 222:15 +CPU= 62.383
  16:31:28 STEP  890 H= 222:30 +CPU= 55.504
  16:32:25 STEP  891 H= 222:45 +CPU= 54.644
  16:33:21 STEP  892 H= 223:00 +CPU= 54.605
  16:34:16 STEP  893 H= 223:15 +CPU= 53.521
  16:35:17 STEP  894 H= 223:30 +CPU= 59.268
  16:36:22 STEP  895 H= 223:45 +CPU= 62.942
  16:37:29 STEP  896 H= 224:00 +CPU= 65.329
  16:38:28 STEP  897 H= 224:15 +CPU= 57.177
  16:39:30 STEP  898 H= 224:30 +CPU= 60.393
  16:40:23 STEP  899 H= 224:45 +CPU= 51.617
  16:41:34 STEP  900 H= 225:00 +CPU= 69.543
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000900
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000900
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000900
  16:42:25 STEP  901 H= 225:15 +CPU= 49.761
  16:43:23 STEP  902 H= 225:30 +CPU= 55.814
  16:44:18 STEP  903 H= 225:45 +CPU= 53.719
  16:45:25 STEP  904 H= 226:00 +CPU= 65.088
  16:46:25 STEP  905 H= 226:15 +CPU= 58.047
  16:47:31 STEP  906 H= 226:30 +CPU= 64.065
  16:48:29 STEP  907 H= 226:45 +CPU= 56.838
  16:49:31 STEP  908 H= 227:00 +CPU= 59.734
  16:50:23 STEP  909 H= 227:15 +CPU= 51.086
  16:51:20 STEP  910 H= 227:30 +CPU= 55.257
  16:52:16 STEP  911 H= 227:45 +CPU= 54.701
  16:53:40 STEP  912 H= 228:00 +CPU= 81.463
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000912
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000912
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000912
  16:54:37 STEP  913 H= 228:15 +CPU= 54.686
  16:55:41 STEP  914 H= 228:30 +CPU= 62.481
  16:56:41 STEP  915 H= 228:45 +CPU= 58.276
  16:57:44 STEP  916 H= 229:00 +CPU= 61.029
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:58:58 STEP  917 H= 229:15 +CPU= 51.341
  16:59:47 STEP  918 H= 229:30 +CPU= 48.285
  17:00:45 STEP  919 H= 229:45 +CPU= 55.885
  17:01:38 STEP  920 H= 230:00 +CPU= 51.942
  17:02:36 STEP  921 H= 230:15 +CPU= 55.669
  17:03:29 STEP  922 H= 230:30 +CPU= 51.707
  17:04:25 STEP  923 H= 230:45 +CPU= 54.942
  17:05:31 STEP  924 H= 231:00 +CPU= 62.258
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000924
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000924
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000924
  17:06:29 STEP  925 H= 231:15 +CPU= 56.514
  17:07:19 STEP  926 H= 231:30 +CPU= 48.368
  17:08:17 STEP  927 H= 231:45 +CPU= 56.630
  17:09:09 STEP  928 H= 232:00 +CPU= 50.733
  17:10:06 STEP  929 H= 232:15 +CPU= 55.618
  17:10:57 STEP  930 H= 232:30 +CPU= 48.919
  17:11:51 STEP  931 H= 232:45 +CPU= 52.680
  17:12:46 STEP  932 H= 233:00 +CPU= 53.944
  17:13:40 STEP  933 H= 233:15 +CPU= 51.983
  17:14:37 STEP  934 H= 233:30 +CPU= 55.538
  17:15:28 STEP  935 H= 233:45 +CPU= 49.343
  17:16:37 STEP  936 H= 234:00 +CPU= 67.679
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000936
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000936
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000936
  17:17:32 STEP  937 H= 234:15 +CPU= 52.934
  17:18:28 STEP  938 H= 234:30 +CPU= 54.443
  17:19:18 STEP  939 H= 234:45 +CPU= 49.430
  17:20:20 STEP  940 H= 235:00 +CPU= 59.288
  17:21:09 STEP  941 H= 235:15 +CPU= 47.616
  17:22:08 STEP  942 H= 235:30 +CPU= 57.844
  17:22:58 STEP  943 H= 235:45 +CPU= 48.712
  17:23:59 STEP  944 H= 236:00 +CPU= 59.464
  17:24:50 STEP  945 H= 236:15 +CPU= 49.391
  17:25:46 STEP  946 H= 236:30 +CPU= 54.766
  17:26:40 STEP  947 H= 236:45 +CPU= 52.372
  17:27:51 STEP  948 H= 237:00 +CPU= 68.912
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000948
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000948
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000948
  17:28:41 STEP  949 H= 237:15 +CPU= 48.791
  17:29:35 STEP  950 H= 237:30 +CPU= 52.941
  17:30:28 STEP  951 H= 237:45 +CPU= 51.602
  17:31:27 STEP  952 H= 238:00 +CPU= 56.677
  17:32:22 STEP  953 H= 238:15 +CPU= 53.496
  17:33:18 STEP  954 H= 238:30 +CPU= 54.225
  17:34:14 STEP  955 H= 238:45 +CPU= 54.720
  17:35:09 STEP  956 H= 239:00 +CPU= 53.442
  17:36:04 STEP  957 H= 239:15 +CPU= 53.711
  17:36:55 STEP  958 H= 239:30 +CPU= 50.049
  17:37:56 STEP  959 H= 239:45 +CPU= 58.775
  17:39:08 STEP  960 H= 240:00 +CPU= 69.144
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000960
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000960
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000960
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000864
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000864
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000864
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000876
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000876
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000876
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000888
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000888
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000888
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000900
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000900
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000900
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000912
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000912
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000912
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000924
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000924
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000924
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000936
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000936
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000936
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000948
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000948
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000948
Zipping up the intermediate file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_9.zip
Uploading the intermediate file: upload_file_9.zip
Uploading trickle at timestep: 864000
  17:40:04 STEP  961 H= 240:15 +CPU= 51.766
  17:40:56 STEP  962 H= 240:30 +CPU= 50.229
  17:41:53 STEP  963 H= 240:45 +CPU= 56.112
  17:42:47 STEP  964 H= 241:00 +CPU= 52.001
  17:43:46 STEP  965 H= 241:15 +CPU= 57.457
  17:44:37 STEP  966 H= 241:30 +CPU= 50.042
  17:45:36 STEP  967 H= 241:45 +CPU= 57.551
  17:46:28 STEP  968 H= 242:00 +CPU= 50.552
  17:47:28 STEP  969 H= 242:15 +CPU= 57.938
  17:48:19 STEP  970 H= 242:30 +CPU= 50.213
  17:49:17 STEP  971 H= 242:45 +CPU= 55.500
  17:50:22 STEP  972 H= 243:00 +CPU= 63.770
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000972
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000972
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000972
  17:51:21 STEP  973 H= 243:15 +CPU= 57.124
  17:52:12 STEP  974 H= 243:30 +CPU= 50.103
  17:53:10 STEP  975 H= 243:45 +CPU= 56.414
  17:54:03 STEP  976 H= 244:00 +CPU= 51.101
  17:55:00 STEP  977 H= 244:15 +CPU= 55.165
  17:55:52 STEP  978 H= 244:30 +CPU= 50.998
  17:56:51 STEP  979 H= 244:45 +CPU= 56.946
  17:57:42 STEP  980 H= 245:00 +CPU= 49.974
  17:58:40 STEP  981 H= 245:15 +CPU= 56.775
  17:59:33 STEP  982 H= 245:30 +CPU= 50.982
  18:00:27 STEP  983 H= 245:45 +CPU= 53.147
  18:01:35 STEP  984 H= 246:00 +CPU= 66.143
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000984
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000984
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000984
  18:02:35 STEP  985 H= 246:15 +CPU= 58.077
  18:03:27 STEP  986 H= 246:30 +CPU= 50.396
  18:04:21 STEP  987 H= 246:45 +CPU= 52.553
  18:05:18 STEP  988 H= 247:00 +CPU= 55.125
  18:06:12 STEP  989 H= 247:15 +CPU= 52.130
  18:07:12 STEP  990 H= 247:30 +CPU= 58.593
  18:08:04 STEP  991 H= 247:45 +CPU= 50.186
  18:09:02 STEP  992 H= 248:00 +CPU= 56.918
  18:09:50 STEP  993 H= 248:15 +CPU= 46.575
  18:10:52 STEP  994 H= 248:30 +CPU= 59.937
  18:11:40 STEP  995 H= 248:45 +CPU= 46.977
  18:12:57 STEP  996 H= 249:00 +CPU= 75.140
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+000996
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+000996
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+000996
  18:13:45 STEP  997 H= 249:15 +CPU= 46.430
  18:14:47 STEP  998 H= 249:30 +CPU= 60.893
  18:15:37 STEP  999 H= 249:45 +CPU= 48.034
  18:16:39 STEP 1000 H= 250:00 +CPU= 60.639
  18:17:28 STEP 1001 H= 250:15 +CPU= 47.576
  18:18:27 STEP 1002 H= 250:30 +CPU= 57.964
  18:19:19 STEP 1003 H= 250:45 +CPU= 50.776
  18:20:19 STEP 1004 H= 251:00 +CPU= 58.706
  18:21:12 STEP 1005 H= 251:15 +CPU= 51.612
  18:22:10 STEP 1006 H= 251:30 +CPU= 55.865
  18:23:04 STEP 1007 H= 251:45 +CPU= 52.679
  18:24:21 STEP 1008 H= 252:00 +CPU= 73.925
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001008
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001008
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001008
  18:25:11 STEP 1009 H= 252:15 +CPU= 48.900
  18:26:09 STEP 1010 H= 252:30 +CPU= 56.129
  18:27:03 STEP 1011 H= 252:45 +CPU= 53.505
  18:28:00 STEP 1012 H= 253:00 +CPU= 54.989
  18:28:59 STEP 1013 H= 253:15 +CPU= 57.246
  18:29:49 STEP 1014 H= 253:30 +CPU= 49.179
  18:30:50 STEP 1015 H= 253:45 +CPU= 58.829
  18:31:39 STEP 1016 H= 254:00 +CPU= 48.005
  18:32:38 STEP 1017 H= 254:15 +CPU= 57.806
  18:33:27 STEP 1018 H= 254:30 +CPU= 47.328
  18:34:28 STEP 1019 H= 254:45 +CPU= 59.338
  18:35:32 STEP 1020 H= 255:00 +CPU= 62.235
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001020
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001020
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001020
  18:36:32 STEP 1021 H= 255:15 +CPU= 58.462
  18:37:22 STEP 1022 H= 255:30 +CPU= 48.816
  18:38:25 STEP 1023 H= 255:45 +CPU= 60.550
  18:39:14 STEP 1024 H= 256:00 +CPU= 48.305
  18:40:15 STEP 1025 H= 256:15 +CPU= 58.915
  18:41:05 STEP 1026 H= 256:30 +CPU= 48.490
  18:42:07 STEP 1027 H= 256:45 +CPU= 60.073
  18:43:00 STEP 1028 H= 257:00 +CPU= 51.725
  18:43:59 STEP 1029 H= 257:15 +CPU= 58.024
  18:44:53 STEP 1030 H= 257:30 +CPU= 52.190
  18:45:51 STEP 1031 H= 257:45 +CPU= 56.652
  18:47:00 STEP 1032 H= 258:00 +CPU= 66.609
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001032
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001032
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001032
  18:48:00 STEP 1033 H= 258:15 +CPU= 58.927
  18:48:52 STEP 1034 H= 258:30 +CPU= 50.175
  18:49:49 STEP 1035 H= 258:45 +CPU= 55.552
  18:50:42 STEP 1036 H= 259:00 +CPU= 51.574
  18:51:38 STEP 1037 H= 259:15 +CPU= 54.705
  18:52:32 STEP 1038 H= 259:30 +CPU= 52.858
  18:53:27 STEP 1039 H= 259:45 +CPU= 53.139
  18:54:26 STEP 1040 H= 260:00 +CPU= 57.593
  18:55:20 STEP 1041 H= 260:15 +CPU= 52.608
  18:56:18 STEP 1042 H= 260:30 +CPU= 56.222
  18:57:11 STEP 1043 H= 260:45 +CPU= 51.269
  18:58:19 STEP 1044 H= 261:00 +CPU= 66.283
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001044
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001044
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001044
  18:59:14 STEP 1045 H= 261:15 +CPU= 52.990
  19:00:12 STEP 1046 H= 261:30 +CPU= 56.817
  19:01:06 STEP 1047 H= 261:45 +CPU= 52.140
  19:02:04 STEP 1048 H= 262:00 +CPU= 56.825
  19:02:55 STEP 1049 H= 262:15 +CPU= 49.057
  19:03:52 STEP 1050 H= 262:30 +CPU= 55.841
  19:04:40 STEP 1051 H= 262:45 +CPU= 46.893
  19:05:39 STEP 1052 H= 263:00 +CPU= 56.091
  19:06:32 STEP 1053 H= 263:15 +CPU= 50.881
  19:07:31 STEP 1054 H= 263:30 +CPU= 57.703
  19:08:23 STEP 1055 H= 263:45 +CPU= 50.158
  19:09:43 STEP 1056 H= 264:00 +CPU= 77.445
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001056
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001056
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001056
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000960
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000960
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000960
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000972
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000972
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000972
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000984
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000984
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000984
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+000996
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+000996
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+000996
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001008
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001008
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001008
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001020
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001020
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001020
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001032
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001032
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001032
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001044
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001044
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001044
Zipping up the intermediate file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_10.zip
Uploading the intermediate file: upload_file_10.zip
  19:10:33 STEP 1057 H= 264:15 +CPU= 46.736
  19:11:32 STEP 1058 H= 264:30 +CPU= 57.261
  19:12:23 STEP 1059 H= 264:45 +CPU= 50.087
  19:13:23 STEP 1060 H= 265:00 +CPU= 58.296
  19:14:15 STEP 1061 H= 265:15 +CPU= 49.986
  19:15:17 STEP 1062 H= 265:30 +CPU= 59.994
  19:16:06 STEP 1063 H= 265:45 +CPU= 47.987
  19:17:06 STEP 1064 H= 266:00 +CPU= 58.026
  19:17:57 STEP 1065 H= 266:15 +CPU= 49.756
  19:18:55 STEP 1066 H= 266:30 +CPU= 57.288
  19:19:49 STEP 1067 H= 266:45 +CPU= 52.095
  19:21:01 STEP 1068 H= 267:00 +CPU= 70.421
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001068
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001068
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001068
  19:21:52 STEP 1069 H= 267:15 +CPU= 49.975
  19:22:53 STEP 1070 H= 267:30 +CPU= 58.463
  19:23:45 STEP 1071 H= 267:45 +CPU= 51.348
  19:24:41 STEP 1072 H= 268:00 +CPU= 54.490
  19:25:37 STEP 1073 H= 268:15 +CPU= 53.861
  19:26:32 STEP 1074 H= 268:30 +CPU= 53.887
  19:27:28 STEP 1075 H= 268:45 +CPU= 54.963
  19:28:22 STEP 1076 H= 269:00 +CPU= 51.875
  19:29:21 STEP 1077 H= 269:15 +CPU= 57.438
  19:30:10 STEP 1078 H= 269:30 +CPU= 47.911
  19:31:10 STEP 1079 H= 269:45 +CPU= 58.706
  19:32:12 STEP 1080 H= 270:00 +CPU= 60.621
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001080
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001080
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001080
  19:33:14 STEP 1081 H= 270:15 +CPU= 60.337
  19:34:03 STEP 1082 H= 270:30 +CPU= 47.690
  19:35:04 STEP 1083 H= 270:45 +CPU= 59.258
  19:35:52 STEP 1084 H= 271:00 +CPU= 46.685
  19:36:53 STEP 1085 H= 271:15 +CPU= 58.977
  19:37:43 STEP 1086 H= 271:30 +CPU= 48.917
  19:38:41 STEP 1087 H= 271:45 +CPU= 56.015
  19:39:35 STEP 1088 H= 272:00 +CPU= 53.088
  19:40:30 STEP 1089 H= 272:15 +CPU= 53.461
  19:41:23 STEP 1090 H= 272:30 +CPU= 51.108
  19:42:19 STEP 1091 H= 272:45 +CPU= 55.063
  19:43:27 STEP 1092 H= 273:00 +CPU= 66.066
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001092
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001092
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001092
  19:44:20 STEP 1093 H= 273:15 +CPU= 51.545
  19:45:14 STEP 1094 H= 273:30 +CPU= 52.530
  19:46:06 STEP 1095 H= 273:45 +CPU= 50.027
  19:47:06 STEP 1096 H= 274:00 +CPU= 58.229
  19:47:58 STEP 1097 H= 274:15 +CPU= 51.320
  19:48:57 STEP 1098 H= 274:30 +CPU= 56.786
  19:49:46 STEP 1099 H= 274:45 +CPU= 47.691
  19:50:47 STEP 1100 H= 275:00 +CPU= 59.499
  19:51:38 STEP 1101 H= 275:15 +CPU= 49.806
  19:52:36 STEP 1102 H= 275:30 +CPU= 56.005
  19:53:25 STEP 1103 H= 275:45 +CPU= 47.785
  19:54:42 STEP 1104 H= 276:00 +CPU= 74.530
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001104
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001104
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001104
  19:55:34 STEP 1105 H= 276:15 +CPU= 50.543
  19:56:32 STEP 1106 H= 276:30 +CPU= 56.451
  19:57:23 STEP 1107 H= 276:45 +CPU= 49.590
  19:58:23 STEP 1108 H= 277:00 +CPU= 58.190
  19:59:16 STEP 1109 H= 277:15 +CPU= 51.296
  20:00:15 STEP 1110 H= 277:30 +CPU= 57.423
  20:01:08 STEP 1111 H= 277:45 +CPU= 51.581
  20:02:04 STEP 1112 H= 278:00 +CPU= 54.223
  20:02:59 STEP 1113 H= 278:15 +CPU= 53.567
  20:03:52 STEP 1114 H= 278:30 +CPU= 51.540
  20:04:49 STEP 1115 H= 278:45 +CPU= 55.554
  20:05:58 STEP 1116 H= 279:00 +CPU= 65.660
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001116
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001116
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001116
  20:06:53 STEP 1117 H= 279:15 +CPU= 53.512
  20:07:45 STEP 1118 H= 279:30 +CPU= 51.133
  20:08:41 STEP 1119 H= 279:45 +CPU= 54.198
  20:09:37 STEP 1120 H= 280:00 +CPU= 54.133
  20:10:33 STEP 1121 H= 280:15 +CPU= 54.631
  20:11:28 STEP 1122 H= 280:30 +CPU= 53.679
  20:12:24 STEP 1123 H= 280:45 +CPU= 54.581
  20:13:17 STEP 1124 H= 281:00 +CPU= 51.743
  20:14:12 STEP 1125 H= 281:15 +CPU= 53.416
  20:15:06 STEP 1126 H= 281:30 +CPU= 52.469
  20:16:03 STEP 1127 H= 281:45 +CPU= 55.715
  20:17:08 STEP 1128 H= 282:00 +CPU= 63.531
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001128
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001128
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001128
  20:18:05 STEP 1129 H= 282:15 +CPU= 55.332
  20:18:57 STEP 1130 H= 282:30 +CPU= 50.348
  20:19:53 STEP 1131 H= 282:45 +CPU= 54.631
  20:20:48 STEP 1132 H= 283:00 +CPU= 53.312
  20:21:43 STEP 1133 H= 283:15 +CPU= 52.851
  20:22:39 STEP 1134 H= 283:30 +CPU= 54.620
  20:23:33 STEP 1135 H= 283:45 +CPU= 52.349
  20:24:28 STEP 1136 H= 284:00 +CPU= 53.983
  20:25:24 STEP 1137 H= 284:15 +CPU= 54.459
  20:26:18 STEP 1138 H= 284:30 +CPU= 52.669
  20:27:13 STEP 1139 H= 284:45 +CPU= 53.939
  20:28:22 STEP 1140 H= 285:00 +CPU= 66.548
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001140
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001140
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001140
  20:29:15 STEP 1141 H= 285:15 +CPU= 52.266
  20:30:09 STEP 1142 H= 285:30 +CPU= 52.438
  20:31:02 STEP 1143 H= 285:45 +CPU= 51.583
  20:31:59 STEP 1144 H= 286:00 +CPU= 55.422
  20:32:54 STEP 1145 H= 286:15 +CPU= 52.978
  20:33:48 STEP 1146 H= 286:30 +CPU= 52.588
  20:34:43 STEP 1147 H= 286:45 +CPU= 53.539
  20:35:35 STEP 1148 H= 287:00 +CPU= 51.077
  20:36:31 STEP 1149 H= 287:15 +CPU= 54.590
  20:37:24 STEP 1150 H= 287:30 +CPU= 51.640
  20:38:21 STEP 1151 H= 287:45 +CPU= 54.896
  20:39:35 STEP 1152 H= 288:00 +CPU= 71.241
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001152
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001152
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001152
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001056
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001056
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001056
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001068
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001068
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001068
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001080
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001080
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001080
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001092
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001092
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001092
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001104
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001104
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001104
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001116
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001116
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001116
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001128
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001128
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001128
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001140
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001140
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001140
Zipping up the intermediate file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_11.zip
Uploading the intermediate file: upload_file_11.zip
  20:40:30 STEP 1153 H= 288:15 +CPU= 52.222
  20:41:20 STEP 1154 H= 288:30 +CPU= 48.539
  20:42:18 STEP 1155 H= 288:45 +CPU= 55.907
  20:43:13 STEP 1156 H= 289:00 +CPU= 52.923
  20:44:06 STEP 1157 H= 289:15 +CPU= 52.014
  20:45:03 STEP 1158 H= 289:30 +CPU= 55.417
  20:45:57 STEP 1159 H= 289:45 +CPU= 52.318
  20:46:55 STEP 1160 H= 290:00 +CPU= 56.539
  20:47:48 STEP 1161 H= 290:15 +CPU= 51.557
  20:48:45 STEP 1162 H= 290:30 +CPU= 54.994
  20:49:36 STEP 1163 H= 290:45 +CPU= 50.458
  20:50:47 STEP 1164 H= 291:00 +CPU= 68.445
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001164
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001164
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001164
  20:51:39 STEP 1165 H= 291:15 +CPU= 50.336
  20:52:35 STEP 1166 H= 291:30 +CPU= 54.282
  20:53:27 STEP 1167 H= 291:45 +CPU= 50.939
  20:54:25 STEP 1168 H= 292:00 +CPU= 56.455
  20:55:18 STEP 1169 H= 292:15 +CPU= 51.954
  20:56:15 STEP 1170 H= 292:30 +CPU= 54.839
  20:57:06 STEP 1171 H= 292:45 +CPU= 49.667
  20:58:03 STEP 1172 H= 293:00 +CPU= 55.813
  20:58:55 STEP 1173 H= 293:15 +CPU= 50.102
  20:59:50 STEP 1174 H= 293:30 +CPU= 53.793
  21:00:45 STEP 1175 H= 293:45 +CPU= 53.074
  21:01:51 STEP 1176 H= 294:00 +CPU= 64.661
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001176
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001176
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001176
  21:02:46 STEP 1177 H= 294:15 +CPU= 53.617
  21:03:38 STEP 1178 H= 294:30 +CPU= 50.864
  21:04:33 STEP 1179 H= 294:45 +CPU= 53.077
  21:05:27 STEP 1180 H= 295:00 +CPU= 51.526
  21:06:22 STEP 1181 H= 295:15 +CPU= 53.367
  21:07:16 STEP 1182 H= 295:30 +CPU= 51.839
  21:08:13 STEP 1183 H= 295:45 +CPU= 55.860
  21:09:08 STEP 1184 H= 296:00 +CPU= 52.868
  21:10:06 STEP 1185 H= 296:15 +CPU= 56.530
  21:10:58 STEP 1186 H= 296:30 +CPU= 50.799
  21:11:55 STEP 1187 H= 296:45 +CPU= 54.815
  21:13:02 STEP 1188 H= 297:00 +CPU= 65.910
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001188
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001188
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001188
  21:13:57 STEP 1189 H= 297:15 +CPU= 53.014
  21:14:50 STEP 1190 H= 297:30 +CPU= 51.942
  21:15:45 STEP 1191 H= 297:45 +CPU= 53.575
  21:16:41 STEP 1192 H= 298:00 +CPU= 53.800
  21:17:35 STEP 1193 H= 298:15 +CPU= 53.234
  21:18:30 STEP 1194 H= 298:30 +CPU= 53.117
  21:19:22 STEP 1195 H= 298:45 +CPU= 50.763
  21:20:20 STEP 1196 H= 299:00 +CPU= 55.973
  21:21:13 STEP 1197 H= 299:15 +CPU= 51.668
  21:22:08 STEP 1198 H= 299:30 +CPU= 53.806
  21:23:02 STEP 1199 H= 299:45 +CPU= 52.778
  21:24:20 STEP 1200 H= 300:00 +CPU= 74.692
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001200
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001200
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001200
  21:25:14 STEP 1201 H= 300:15 +CPU= 52.925
  21:26:12 STEP 1202 H= 300:30 +CPU= 55.942
  21:27:05 STEP 1203 H= 300:45 +CPU= 52.383
  21:28:02 STEP 1204 H= 301:00 +CPU= 54.443
  21:28:55 STEP 1205 H= 301:15 +CPU= 51.665
  21:29:48 STEP 1206 H= 301:30 +CPU= 52.270
  21:30:45 STEP 1207 H= 301:45 +CPU= 54.601
  21:31:38 STEP 1208 H= 302:00 +CPU= 52.048
  21:32:34 STEP 1209 H= 302:15 +CPU= 54.343
  21:33:27 STEP 1210 H= 302:30 +CPU= 51.439
  21:34:25 STEP 1211 H= 302:45 +CPU= 56.478
  21:35:34 STEP 1212 H= 303:00 +CPU= 66.740
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001212
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001212
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001212
  21:36:31 STEP 1213 H= 303:15 +CPU= 55.914
  21:37:23 STEP 1214 H= 303:30 +CPU= 50.872
  21:38:22 STEP 1215 H= 303:45 +CPU= 57.043
  21:39:16 STEP 1216 H= 304:00 +CPU= 52.678
  21:40:12 STEP 1217 H= 304:15 +CPU= 53.867
  21:41:05 STEP 1218 H= 304:30 +CPU= 51.677
  21:42:00 STEP 1219 H= 304:45 +CPU= 53.387
  21:42:52 STEP 1220 H= 305:00 +CPU= 51.061
  21:43:49 STEP 1221 H= 305:15 +CPU= 55.053
  21:44:42 STEP 1222 H= 305:30 +CPU= 51.903
  21:45:39 STEP 1223 H= 305:45 +CPU= 54.763
  21:46:48 STEP 1224 H= 306:00 +CPU= 67.258
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001224
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001224
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001224
  21:47:40 STEP 1225 H= 306:15 +CPU= 50.857
  21:48:36 STEP 1226 H= 306:30 +CPU= 54.162
  21:49:29 STEP 1227 H= 306:45 +CPU= 51.631
  21:50:29 STEP 1228 H= 307:00 +CPU= 58.686
  21:51:22 STEP 1229 H= 307:15 +CPU= 51.556
  21:52:20 STEP 1230 H= 307:30 +CPU= 56.324
  21:53:10 STEP 1231 H= 307:45 +CPU= 48.513
  21:54:10 STEP 1232 H= 308:00 +CPU= 59.176
  21:55:01 STEP 1233 H= 308:15 +CPU= 49.436
  21:56:00 STEP 1234 H= 308:30 +CPU= 56.957
  21:56:50 STEP 1235 H= 308:45 +CPU= 49.475
  21:58:02 STEP 1236 H= 309:00 +CPU= 69.293
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001236
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001236
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001236
  21:58:53 STEP 1237 H= 309:15 +CPU= 49.768
  21:59:52 STEP 1238 H= 309:30 +CPU= 57.846
  22:00:45 STEP 1239 H= 309:45 +CPU= 51.741
  22:01:45 STEP 1240 H= 310:00 +CPU= 58.466
  22:02:37 STEP 1241 H= 310:15 +CPU= 50.941
  22:03:32 STEP 1242 H= 310:30 +CPU= 53.500
  22:04:27 STEP 1243 H= 310:45 +CPU= 53.509
  22:05:23 STEP 1244 H= 311:00 +CPU= 52.402
  22:06:21 STEP 1245 H= 311:15 +CPU= 54.275
  22:07:14 STEP 1246 H= 311:30 +CPU= 50.977
  22:08:10 STEP 1247 H= 311:45 +CPU= 54.199
  22:09:22 STEP 1248 H= 312:00 +CPU= 69.988
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001248
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001248
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001248
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001152
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001152
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001152
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001164
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001164
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001164
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001176
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001176
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001176
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001188
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001188
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001188
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001200
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001200
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001200
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001212
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001212
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001212
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001224
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001224
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001224
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001236
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001236
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001236
Zipping up the intermediate file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_12.zip
Uploading the intermediate file: upload_file_12.zip
  22:10:20 STEP 1249 H= 312:15 +CPU= 53.634
  22:11:11 STEP 1250 H= 312:30 +CPU= 49.388
  22:12:09 STEP 1251 H= 312:45 +CPU= 55.900
  22:13:00 STEP 1252 H= 313:00 +CPU= 50.094
  22:13:57 STEP 1253 H= 313:15 +CPU= 55.633
  22:14:49 STEP 1254 H= 313:30 +CPU= 50.253
  22:15:50 STEP 1255 H= 313:45 +CPU= 59.504
  22:16:40 STEP 1256 H= 314:00 +CPU= 48.851
  22:17:42 STEP 1257 H= 314:15 +CPU= 60.175
  22:18:30 STEP 1258 H= 314:30 +CPU= 46.608
  22:19:29 STEP 1259 H= 314:45 +CPU= 57.532
  22:20:31 STEP 1260 H= 315:00 +CPU= 60.741
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001260
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001260
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001260
  22:21:29 STEP 1261 H= 315:15 +CPU= 55.921
  22:22:20 STEP 1262 H= 315:30 +CPU= 49.517
  22:23:19 STEP 1263 H= 315:45 +CPU= 57.945
  22:24:13 STEP 1264 H= 316:00 +CPU= 52.413
  22:25:10 STEP 1265 H= 316:15 +CPU= 55.092
  22:26:05 STEP 1266 H= 316:30 +CPU= 53.822
  22:26:57 STEP 1267 H= 316:45 +CPU= 51.020
  22:27:54 STEP 1268 H= 317:00 +CPU= 55.649
  22:28:48 STEP 1269 H= 317:15 +CPU= 51.775
  22:29:43 STEP 1270 H= 317:30 +CPU= 54.432
  22:30:36 STEP 1271 H= 317:45 +CPU= 51.558
  22:31:49 STEP 1272 H= 318:00 +CPU= 70.365
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001272
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001272
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001272
  22:32:43 STEP 1273 H= 318:15 +CPU= 52.452
  22:33:38 STEP 1274 H= 318:30 +CPU= 53.488
  22:34:35 STEP 1275 H= 318:45 +CPU= 56.283
  22:35:33 STEP 1276 H= 319:00 +CPU= 56.177
  22:36:26 STEP 1277 H= 319:15 +CPU= 51.186
  22:37:22 STEP 1278 H= 319:30 +CPU= 54.122
  22:38:13 STEP 1279 H= 319:45 +CPU= 50.270
  22:39:14 STEP 1280 H= 320:00 +CPU= 59.150
  22:40:03 STEP 1281 H= 320:15 +CPU= 48.380
  22:41:03 STEP 1282 H= 320:30 +CPU= 58.232
  22:41:54 STEP 1283 H= 320:45 +CPU= 49.161
  22:43:07 STEP 1284 H= 321:00 +CPU= 71.518
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001284
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001284
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001284
  22:44:00 STEP 1285 H= 321:15 +CPU= 51.020
  22:44:59 STEP 1286 H= 321:30 +CPU= 57.786
  22:45:50 STEP 1287 H= 321:45 +CPU= 49.786
  22:46:50 STEP 1288 H= 322:00 +CPU= 57.884
  22:47:41 STEP 1289 H= 322:15 +CPU= 50.151
  22:48:39 STEP 1290 H= 322:30 +CPU= 56.270
  22:49:34 STEP 1291 H= 322:45 +CPU= 53.804
  22:50:34 STEP 1292 H= 323:00 +CPU= 58.362
  22:51:25 STEP 1293 H= 323:15 +CPU= 49.536
  22:52:26 STEP 1294 H= 323:30 +CPU= 59.424
  22:53:18 STEP 1295 H= 323:45 +CPU= 50.459
  22:54:37 STEP 1296 H= 324:00 +CPU= 76.584
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001296
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001296
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001296
  22:55:30 STEP 1297 H= 324:15 +CPU= 51.271
  22:56:27 STEP 1298 H= 324:30 +CPU= 55.348
  22:57:18 STEP 1299 H= 324:45 +CPU= 49.651
  22:58:18 STEP 1300 H= 325:00 +CPU= 58.843
  22:59:09 STEP 1301 H= 325:15 +CPU= 49.595
  23:00:08 STEP 1302 H= 325:30 +CPU= 57.202
  23:01:02 STEP 1303 H= 325:45 +CPU= 52.664
  23:02:01 STEP 1304 H= 326:00 +CPU= 56.990
  23:02:53 STEP 1305 H= 326:15 +CPU= 51.191
  23:03:48 STEP 1306 H= 326:30 +CPU= 53.637
  23:04:46 STEP 1307 H= 326:45 +CPU= 55.955
  23:05:54 STEP 1308 H= 327:00 +CPU= 65.322
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001308
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001308
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001308
  23:06:48 STEP 1309 H= 327:15 +CPU= 52.311
  23:07:44 STEP 1310 H= 327:30 +CPU= 54.061
  23:08:41 STEP 1311 H= 327:45 +CPU= 55.913
  23:09:38 STEP 1312 H= 328:00 +CPU= 55.585
  23:10:35 STEP 1313 H= 328:15 +CPU= 55.673
  23:11:28 STEP 1314 H= 328:30 +CPU= 50.981
  23:12:25 STEP 1315 H= 328:45 +CPU= 56.250
  23:13:17 STEP 1316 H= 329:00 +CPU= 50.398
  23:14:14 STEP 1317 H= 329:15 +CPU= 55.511
  23:15:04 STEP 1318 H= 329:30 +CPU= 47.974
  23:16:02 STEP 1319 H= 329:45 +CPU= 56.377
  23:17:08 STEP 1320 H= 330:00 +CPU= 63.594
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001320
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001320
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001320
  23:18:07 STEP 1321 H= 330:15 +CPU= 56.377
  23:19:02 STEP 1322 H= 330:30 +CPU= 52.496
  23:20:02 STEP 1323 H= 330:45 +CPU= 58.596
  23:20:56 STEP 1324 H= 331:00 +CPU= 52.118
  23:21:56 STEP 1325 H= 331:15 +CPU= 58.076
  23:22:51 STEP 1326 H= 331:30 +CPU= 53.067
  23:23:53 STEP 1327 H= 331:45 +CPU= 59.791
  23:24:50 STEP 1328 H= 332:00 +CPU= 55.723
  23:25:49 STEP 1329 H= 332:15 +CPU= 57.185
  23:26:43 STEP 1330 H= 332:30 +CPU= 52.381
  23:27:42 STEP 1331 H= 332:45 +CPU= 56.603
  23:28:51 STEP 1332 H= 333:00 +CPU= 67.143
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001332
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001332
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001332
  23:29:51 STEP 1333 H= 333:15 +CPU= 58.471
  23:30:46 STEP 1334 H= 333:30 +CPU= 52.932
  23:31:46 STEP 1335 H= 333:45 +CPU= 58.547
  23:32:42 STEP 1336 H= 334:00 +CPU= 53.730
  23:33:40 STEP 1337 H= 334:15 +CPU= 55.694
  23:34:35 STEP 1338 H= 334:30 +CPU= 53.859
  23:35:30 STEP 1339 H= 334:45 +CPU= 53.332
  23:36:27 STEP 1340 H= 335:00 +CPU= 54.991
  23:37:22 STEP 1341 H= 335:15 +CPU= 53.592
  23:38:19 STEP 1342 H= 335:30 +CPU= 55.346
  23:39:14 STEP 1343 H= 335:45 +CPU= 53.213
  23:40:30 STEP 1344 H= 336:00 +CPU= 73.057
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001344
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001344
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001344
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001248
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001248
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001248
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001260
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001260
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001260
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001272
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001272
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001272
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001284
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001284
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001284
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001296
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001296
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001296
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001308
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001308
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001308
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001320
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001320
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001320
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001332
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001332
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001332
Zipping up the intermediate file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_13.zip
Uploading the intermediate file: upload_file_13.zip
  23:41:28 STEP 1345 H= 336:15 +CPU= 54.378
  23:42:23 STEP 1346 H= 336:30 +CPU= 52.826
  23:43:21 STEP 1347 H= 336:45 +CPU= 55.675
  23:44:18 STEP 1348 H= 337:00 +CPU= 54.843
  23:45:12 STEP 1349 H= 337:15 +CPU= 51.998
  23:46:10 STEP 1350 H= 337:30 +CPU= 55.458
  23:47:02 STEP 1351 H= 337:45 +CPU= 50.433
  23:48:00 STEP 1352 H= 338:00 +CPU= 57.124
  23:48:55 STEP 1353 H= 338:15 +CPU= 53.259
  23:49:51 STEP 1354 H= 338:30 +CPU= 54.457
  23:50:49 STEP 1355 H= 338:45 +CPU= 55.221
  23:51:56 STEP 1356 H= 339:00 +CPU= 65.468
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001356
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001356
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001356
  23:52:51 STEP 1357 H= 339:15 +CPU= 53.009
  23:53:47 STEP 1358 H= 339:30 +CPU= 55.027
  23:54:44 STEP 1359 H= 339:45 +CPU= 54.705
  23:55:41 STEP 1360 H= 340:00 +CPU= 55.133
  23:56:36 STEP 1361 H= 340:15 +CPU= 53.308
  23:57:31 STEP 1362 H= 340:30 +CPU= 53.658
  23:58:25 STEP 1363 H= 340:45 +CPU= 52.163
  23:59:23 STEP 1364 H= 341:00 +CPU= 55.998
  00:00:19 STEP 1365 H= 341:15 +CPU= 54.343
  00:01:06 STEP 1366 H= 341:30 +CPU= 46.053
  00:01:52 STEP 1367 H= 341:45 +CPU= 44.745
  00:02:56 STEP 1368 H= 342:00 +CPU= 62.073
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001368
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001368
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001368
  00:03:42 STEP 1369 H= 342:15 +CPU= 45.033
  00:04:30 STEP 1370 H= 342:30 +CPU= 46.546
  00:05:17 STEP 1371 H= 342:45 +CPU= 45.322
  00:06:03 STEP 1372 H= 343:00 +CPU= 44.732
  00:06:48 STEP 1373 H= 343:15 +CPU= 43.542
  00:07:36 STEP 1374 H= 343:30 +CPU= 46.830
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  00:08:43 STEP 1375 H= 343:45 +CPU= 45.168
  00:09:33 STEP 1376 H= 344:00 +CPU= 48.247
  00:10:25 STEP 1377 H= 344:15 +CPU= 50.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:14:00 STEP 1378 H= 344:30 +CPU= 30.492
  03:14:25 STEP 1379 H= 344:45 +CPU= 24.870
  03:14:56 STEP 1380 H= 345:00 +CPU= 30.385
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001380
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001380
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001380
  03:15:20 STEP 1381 H= 345:15 +CPU= 23.837
  03:15:44 STEP 1382 H= 345:30 +CPU= 23.931
  03:16:09 STEP 1383 H= 345:45 +CPU= 23.730
  03:16:33 STEP 1384 H= 346:00 +CPU= 24.453
  03:16:58 STEP 1385 H= 346:15 +CPU= 24.150
  03:17:22 STEP 1386 H= 346:30 +CPU= 24.154
  03:17:47 STEP 1387 H= 346:45 +CPU= 24.190
  03:18:12 STEP 1388 H= 347:00 +CPU= 24.861
  03:18:37 STEP 1389 H= 347:15 +CPU= 24.231
  03:19:01 STEP 1390 H= 347:30 +CPU= 24.219
  03:19:26 STEP 1391 H= 347:45 +CPU= 24.153
  03:20:00 STEP 1392 H= 348:00 +CPU= 33.318
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001392
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001392
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001392
  03:20:25 STEP 1393 H= 348:15 +CPU= 24.238
  03:20:49 STEP 1394 H= 348:30 +CPU= 23.948
  03:21:13 STEP 1395 H= 348:45 +CPU= 23.828
  03:21:38 STEP 1396 H= 349:00 +CPU= 24.652
  03:22:03 STEP 1397 H= 349:15 +CPU= 24.115
  03:22:27 STEP 1398 H= 349:30 +CPU= 23.818
  03:22:53 STEP 1399 H= 349:45 +CPU= 25.475
  03:23:19 STEP 1400 H= 350:00 +CPU= 25.569
  03:23:44 STEP 1401 H= 350:15 +CPU= 24.992
  03:24:09 STEP 1402 H= 350:30 +CPU= 23.855
  03:24:33 STEP 1403 H= 350:45 +CPU= 24.063
  03:25:04 STEP 1404 H= 351:00 +CPU= 30.416
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001404
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001404
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001404
  03:25:29 STEP 1405 H= 351:15 +CPU= 24.520
  03:25:54 STEP 1406 H= 351:30 +CPU= 24.749
  03:26:19 STEP 1407 H= 351:45 +CPU= 24.173
  03:26:43 STEP 1408 H= 352:00 +CPU= 24.638
  03:27:08 STEP 1409 H= 352:15 +CPU= 24.292
  03:27:33 STEP 1410 H= 352:30 +CPU= 24.390
  03:27:57 STEP 1411 H= 352:45 +CPU= 24.019
  03:28:23 STEP 1412 H= 353:00 +CPU= 25.145
  03:28:47 STEP 1413 H= 353:15 +CPU= 24.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  03:29:33 STEP 1414 H= 353:30 +CPU= 24.216
  03:29:59 STEP 1415 H= 353:45 +CPU= 25.402
  03:30:33 STEP 1416 H= 354:00 +CPU= 33.256
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001416
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001416
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001416
  03:30:59 STEP 1417 H= 354:15 +CPU= 25.787
  03:31:27 STEP 1418 H= 354:30 +CPU= 27.205
  03:31:53 STEP 1419 H= 354:45 +CPU= 25.361
  03:32:19 STEP 1420 H= 355:00 +CPU= 25.823
  03:32:45 STEP 1421 H= 355:15 +CPU= 25.296
  03:33:10 STEP 1422 H= 355:30 +CPU= 25.279
  03:33:36 STEP 1423 H= 355:45 +CPU= 25.105
  03:34:02 STEP 1424 H= 356:00 +CPU= 25.581
  03:34:29 STEP 1425 H= 356:15 +CPU= 25.947
  03:34:56 STEP 1426 H= 356:30 +CPU= 27.046
  03:35:22 STEP 1427 H= 356:45 +CPU= 25.823
  03:35:55 STEP 1428 H= 357:00 +CPU= 32.295
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001428
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001428
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001428
  03:36:21 STEP 1429 H= 357:15 +CPU= 25.411
  03:36:47 STEP 1430 H= 357:30 +CPU= 24.969
  03:37:12 STEP 1431 H= 357:45 +CPU= 24.967
  03:37:38 STEP 1432 H= 358:00 +CPU= 25.731
  03:38:04 STEP 1433 H= 358:15 +CPU= 24.954
  03:38:29 STEP 1434 H= 358:30 +CPU= 25.137
  03:38:55 STEP 1435 H= 358:45 +CPU= 25.358
  03:39:22 STEP 1436 H= 359:00 +CPU= 26.490
  03:39:48 STEP 1437 H= 359:15 +CPU= 24.929
  03:40:13 STEP 1438 H= 359:30 +CPU= 25.098
  03:40:39 STEP 1439 H= 359:45 +CPU= 25.384
  03:41:13 STEP 1440 H= 360:00 +CPU= 32.783
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.2133052439E-21      I       0.2071095843E-05      I
ICloud fractionI      0.0000000000E+00      I       0.6990180716E-01      I
I  Relat. Hum. I      0.3173131565E+00      I       0.4628451736E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.4801046604E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    316747985 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0130( 2.3%)
 03:41:13 STEP 1440 H= 360:00 +CPU= 32.783
 END CNT3
 NSTEP =  1440 CNT0   000000000  23.905  23.905  24.384   0.980               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        42886/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    758    0        0            0    42886 ************    86.5     0.1    86.4     0.1   276.2
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.98      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 1562   27360.2   27360.2     35.23      0.00
   9 SPCM     - SPECTRAL COMP.                    1440    1482.9    1482.9      1.76      0.00
  10 SCAN2M - PHYSICS                             1441   34981.4   34981.4     41.55      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                121    1159.0    1159.0      0.12      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 63     101.5     101.5      0.01      0.00
  14 SUINIF                                          1   28522.3   28522.3      0.02      0.00
  17 GRIDFPOS IN CNT4                              121      60.9      60.9      0.01      0.00
  18 SUSPECG                                         1    3798.9    3798.9      0.00      0.00
  19 SUSPEC                                          1    3802.9    3802.9      0.00      0.00
  24 SUGRIDU                                         1    2376.3    2376.3      0.00      0.00
  25 SPECRT                                          1     878.6     878.6      0.00      0.00
  26 SUGRIDF                                         1   21464.4   21464.4      0.02      0.00
  27 RESTART FILES - WRITING                        30    4674.7    4674.7      0.12      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               121       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                               121   12964.1   12964.1      1.29      0.00
  31 POSDDH IN STEPO                                 7       2.1       2.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1441   10997.0   10997.0     13.06      0.00
  39 SU0YOMB                                         1     709.8     709.8      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1441      74.3      74.3      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1440     819.9     819.9      0.97      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1440     227.2     227.2      0.27      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 63       0.5       0.5      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     3125     831.3     831.3      2.14      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      3006     502.7     502.7      1.25      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       3006     249.5     249.5      0.62      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      3125     271.6     271.6      0.70      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1     129.9     129.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           3125     117.1     117.1      0.30      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           3006      53.3      53.3      0.13      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           3125     160.0     160.0      0.41      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           3006      69.8      69.8      0.17      0.00
 400 GSTATS                                     167374       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                155706       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME   121313.7 CPU TIME   75422.0 VECTOR TIME    75422.0


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       10.7
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/home/karl/slots/2/ICMGGh7zg+001440
Moving to projects directory: /var/home/karl/slots/2/ICMSHh7zg+001440
Moving to projects directory: /var/home/karl/slots/2/ICMUAh7zg+001440
Adding to the zip: /var/home/karl/slots/2/NODE.001_01
Adding to the zip: /var/home/karl/slots/2/ifs.stat
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001344
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001344
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001344
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001356
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001356
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001356
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001368
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001368
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001368
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001380
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001380
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001380
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001392
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001392
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001392
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001404
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001404
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001404
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001416
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001416
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001416
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001428
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001428
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001428
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMGGh7zg+001440
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMSHh7zg+001440
Adding to the zip: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_12213102/ICMUAh7zg+001440
Zipping up the final file: /var/home/karl/projects/climateprediction.net/oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1_r2048191801_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
03:44:46 (192): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:54:57 1528108 22314745 oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1 1,295,100 75,422 0.0582
09 May 2023 16:54:53 1528108 22314745 oifs_43r3_bl_a4vf_2016092300_15_991_12213102_1 864,000 51,029 0.0591


©2024 climateprediction.net