climateprediction.net home page
Task 22287320

Task 22287320

Name oifs_43r3_ps_0404_2015050100_123_984_12201048_0
Workunit 12201048
Created 24 Dec 2022, 12:42:07 UTC
Sent 21 Jan 2023, 21:51:08 UTC
Report deadline 22 Mar 2023, 21:51:08 UTC
Received 23 Jan 2023, 2:32:15 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1537854
Run time 1 days 0 hours 39 min 32 sec
CPU time 23 hours 12 min 43 sec
Validate state Invalid
Credit 7,862.40
Device peak FLOPS 3.60 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.30 MB
Peak swap size 4,978.08 MB
Peak disk usage 3,489.56 MB
Stderr
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ing to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000864
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000876
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000876
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000876
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_36.zip
Uploading the intermediate file: upload_file_36.zip
  14:12:09 STEP  889 H= 889:00 +CPU= 43.234
  14:12:57 STEP  890 H= 890:00 +CPU= 44.748
  14:14:09 STEP  891 H= 891:00 +CPU= 68.120
  14:14:57 STEP  892 H= 892:00 +CPU= 44.879
  14:15:43 STEP  893 H= 893:00 +CPU= 43.509
  14:16:58 STEP  894 H= 894:00 +CPU= 68.907
  14:17:44 STEP  895 H= 895:00 +CPU= 43.502
  14:18:32 STEP  896 H= 896:00 +CPU= 44.598
  14:19:45 STEP  897 H= 897:00 +CPU= 68.626
  14:20:33 STEP  898 H= 898:00 +CPU= 43.767
  14:21:19 STEP  899 H= 899:00 +CPU= 43.441
  14:22:41 STEP  900 H= 900:00 +CPU= 76.820
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000900
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000900
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000900
  14:23:27 STEP  901 H= 901:00 +CPU= 43.530
  14:24:15 STEP  902 H= 902:00 +CPU= 44.894
  14:25:28 STEP  903 H= 903:00 +CPU= 68.373
  14:26:16 STEP  904 H= 904:00 +CPU= 44.838
  14:27:02 STEP  905 H= 905:00 +CPU= 43.629
  14:28:17 STEP  906 H= 906:00 +CPU= 69.816
  14:29:03 STEP  907 H= 907:00 +CPU= 43.533
  14:29:51 STEP  908 H= 908:00 +CPU= 44.793
  14:31:04 STEP  909 H= 909:00 +CPU= 68.224
  14:31:52 STEP  910 H= 910:00 +CPU= 44.870
  14:32:40 STEP  911 H= 911:00 +CPU= 44.968
  14:34:06 STEP  912 H= 912:00 +CPU= 79.838
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000912
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000912
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000888
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000900
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000900
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000900
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_37.zip
Uploading the intermediate file: upload_file_37.zip
  14:34:52 STEP  913 H= 913:00 +CPU= 43.384
  14:35:40 STEP  914 H= 914:00 +CPU= 44.695
  14:36:53 STEP  915 H= 915:00 +CPU= 67.702
  14:37:41 STEP  916 H= 916:00 +CPU= 44.803
  14:38:28 STEP  917 H= 917:00 +CPU= 43.250
  14:39:42 STEP  918 H= 918:00 +CPU= 70.056
  14:40:29 STEP  919 H= 919:00 +CPU= 42.573
  14:41:17 STEP  920 H= 920:00 +CPU= 44.791
  14:42:30 STEP  921 H= 921:00 +CPU= 68.421
  14:43:18 STEP  922 H= 922:00 +CPU= 44.843
  14:44:04 STEP  923 H= 923:00 +CPU= 43.424
  14:45:26 STEP  924 H= 924:00 +CPU= 76.642
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000924
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000924
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000924
  14:46:12 STEP  925 H= 925:00 +CPU= 43.544
  14:47:00 STEP  926 H= 926:00 +CPU= 44.708
  14:48:13 STEP  927 H= 927:00 +CPU= 68.331
  14:49:00 STEP  928 H= 928:00 +CPU= 44.784
  14:49:46 STEP  929 H= 929:00 +CPU= 43.465
  14:51:01 STEP  930 H= 930:00 +CPU= 70.018
  14:51:48 STEP  931 H= 931:00 +CPU= 43.472
  14:52:35 STEP  932 H= 932:00 +CPU= 44.679
  14:53:49 STEP  933 H= 933:00 +CPU= 68.597
  14:54:37 STEP  934 H= 934:00 +CPU= 44.616
  14:55:25 STEP  935 H= 935:00 +CPU= 44.932
  14:56:51 STEP  936 H= 936:00 +CPU= 80.058
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000936
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000936
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000912
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000924
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000924
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000924
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_38.zip
Uploading the intermediate file: upload_file_38.zip
  14:57:38 STEP  937 H= 937:00 +CPU= 43.635
  14:58:26 STEP  938 H= 938:00 +CPU= 44.402
  14:59:39 STEP  939 H= 939:00 +CPU= 68.616
  15:00:27 STEP  940 H= 940:00 +CPU= 44.171
  15:01:13 STEP  941 H= 941:00 +CPU= 43.674
  15:02:28 STEP  942 H= 942:00 +CPU= 70.166
  15:03:15 STEP  943 H= 943:00 +CPU= 43.656
  15:04:03 STEP  944 H= 944:00 +CPU= 44.889
  15:05:16 STEP  945 H= 945:00 +CPU= 68.652
  15:06:04 STEP  946 H= 946:00 +CPU= 45.101
  15:06:51 STEP  947 H= 947:00 +CPU= 43.732
  15:08:13 STEP  948 H= 948:00 +CPU= 76.163
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000948
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000948
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000948
  15:08:59 STEP  949 H= 949:00 +CPU= 43.730
  15:09:47 STEP  950 H= 950:00 +CPU= 44.975
  15:11:00 STEP  951 H= 951:00 +CPU= 68.605
  15:11:48 STEP  952 H= 952:00 +CPU= 44.768
  15:12:34 STEP  953 H= 953:00 +CPU= 43.533
  15:13:49 STEP  954 H= 954:00 +CPU= 69.869
  15:14:36 STEP  955 H= 955:00 +CPU= 43.367
  15:15:24 STEP  956 H= 956:00 +CPU= 44.769
  15:16:37 STEP  957 H= 957:00 +CPU= 67.980
  15:17:25 STEP  958 H= 958:00 +CPU= 44.920
  15:18:13 STEP  959 H= 959:00 +CPU= 44.577
  15:19:39 STEP  960 H= 960:00 +CPU= 80.188
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000960
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000960
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000936
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000948
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000948
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000948
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_39.zip
Uploading the intermediate file: upload_file_39.zip
  15:20:26 STEP  961 H= 961:00 +CPU= 42.229
  15:21:13 STEP  962 H= 962:00 +CPU= 44.654
  15:22:27 STEP  963 H= 963:00 +CPU= 68.688
  15:23:15 STEP  964 H= 964:00 +CPU= 44.784
  15:24:01 STEP  965 H= 965:00 +CPU= 43.420
  15:25:16 STEP  966 H= 966:00 +CPU= 70.073
  15:26:02 STEP  967 H= 967:00 +CPU= 43.418
  15:26:50 STEP  968 H= 968:00 +CPU= 44.747
  15:28:03 STEP  969 H= 969:00 +CPU= 68.376
  15:28:50 STEP  970 H= 970:00 +CPU= 44.860
  15:29:37 STEP  971 H= 971:00 +CPU= 43.507
  15:30:58 STEP  972 H= 972:00 +CPU= 75.730
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000972
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000972
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000972
  15:31:44 STEP  973 H= 973:00 +CPU= 43.570
  15:32:33 STEP  974 H= 974:00 +CPU= 44.637
  15:33:46 STEP  975 H= 975:00 +CPU= 68.521
  15:34:33 STEP  976 H= 976:00 +CPU= 44.374
  15:35:19 STEP  977 H= 977:00 +CPU= 43.433
  15:36:34 STEP  978 H= 978:00 +CPU= 69.354
  15:37:20 STEP  979 H= 979:00 +CPU= 43.462
  15:38:08 STEP  980 H= 980:00 +CPU= 44.333
  15:39:21 STEP  981 H= 981:00 +CPU= 68.546
  15:40:09 STEP  982 H= 982:00 +CPU= 43.962
  15:40:56 STEP  983 H= 983:00 +CPU= 44.991
  15:42:22 STEP  984 H= 984:00 +CPU= 80.239
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000984
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000984
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000960
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000972
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000972
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000972
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_40.zip
Uploading the intermediate file: upload_file_40.zip
  15:43:09 STEP  985 H= 985:00 +CPU= 43.511
  15:43:57 STEP  986 H= 986:00 +CPU= 44.948
  15:45:10 STEP  987 H= 987:00 +CPU= 68.628
  15:45:58 STEP  988 H= 988:00 +CPU= 44.720
  15:46:45 STEP  989 H= 989:00 +CPU= 43.283
  15:48:00 STEP  990 H= 990:00 +CPU= 69.899
  15:48:46 STEP  991 H= 991:00 +CPU= 43.421
  15:49:34 STEP  992 H= 992:00 +CPU= 44.693
  15:50:47 STEP  993 H= 993:00 +CPU= 68.186
  15:51:35 STEP  994 H= 994:00 +CPU= 44.895
  15:52:21 STEP  995 H= 995:00 +CPU= 43.512
  15:53:43 STEP  996 H= 996:00 +CPU= 76.558
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+000996
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+000996
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+000996
  15:54:30 STEP  997 H= 997:00 +CPU= 43.293
  15:55:18 STEP  998 H= 998:00 +CPU= 44.943
  15:56:31 STEP  999 H= 999:00 +CPU= 67.783
  15:57:19 STEP 1000 H=1000:00 +CPU= 44.779
  15:58:06 STEP 1001 H=1001:00 +CPU= 43.042
  15:59:20 STEP 1002 H=1002:00 +CPU= 69.916
  16:00:07 STEP 1003 H=1003:00 +CPU= 42.932
  16:00:54 STEP 1004 H=1004:00 +CPU= 44.663
  16:02:08 STEP 1005 H=1005:00 +CPU= 68.604
  16:02:55 STEP 1006 H=1006:00 +CPU= 44.864
  16:03:43 STEP 1007 H=1007:00 +CPU= 45.052
  16:05:09 STEP 1008 H=1008:00 +CPU= 80.100
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001008
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001008
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000984
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+000996
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+000996
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+000996
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_41.zip
Uploading the intermediate file: upload_file_41.zip
  16:05:56 STEP 1009 H=1009:00 +CPU= 43.431
  16:06:43 STEP 1010 H=1010:00 +CPU= 44.678
  16:07:56 STEP 1011 H=1011:00 +CPU= 68.216
  16:08:44 STEP 1012 H=1012:00 +CPU= 44.579
  16:09:30 STEP 1013 H=1013:00 +CPU= 43.276
  16:10:44 STEP 1014 H=1014:00 +CPU= 69.210
  16:11:30 STEP 1015 H=1015:00 +CPU= 43.508
  16:12:18 STEP 1016 H=1016:00 +CPU= 44.525
  16:13:31 STEP 1017 H=1017:00 +CPU= 68.244
  16:14:19 STEP 1018 H=1018:00 +CPU= 44.107
  16:15:05 STEP 1019 H=1019:00 +CPU= 43.506
  16:16:27 STEP 1020 H=1020:00 +CPU= 75.836
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001020
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001020
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001020
  16:17:14 STEP 1021 H=1021:00 +CPU= 43.449
  16:18:02 STEP 1022 H=1022:00 +CPU= 44.441
  16:19:15 STEP 1023 H=1023:00 +CPU= 68.604
  16:20:03 STEP 1024 H=1024:00 +CPU= 44.396
  16:20:50 STEP 1025 H=1025:00 +CPU= 43.400
  16:22:04 STEP 1026 H=1026:00 +CPU= 69.713
  16:22:50 STEP 1027 H=1027:00 +CPU= 43.486
  16:23:38 STEP 1028 H=1028:00 +CPU= 44.824
  16:24:51 STEP 1029 H=1029:00 +CPU= 68.373
  16:25:39 STEP 1030 H=1030:00 +CPU= 44.663
  16:26:27 STEP 1031 H=1031:00 +CPU= 44.957
  16:27:53 STEP 1032 H=1032:00 +CPU= 80.357
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001032
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001032
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001008
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001020
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001020
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001020
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_42.zip
Uploading the intermediate file: upload_file_42.zip
  16:28:39 STEP 1033 H=1033:00 +CPU= 43.322
  16:29:27 STEP 1034 H=1034:00 +CPU= 44.740
  16:30:41 STEP 1035 H=1035:00 +CPU= 67.758
  16:31:28 STEP 1036 H=1036:00 +CPU= 44.694
  16:32:15 STEP 1037 H=1037:00 +CPU= 43.543
  16:33:29 STEP 1038 H=1038:00 +CPU= 69.831
  16:34:16 STEP 1039 H=1039:00 +CPU= 43.280
  16:35:04 STEP 1040 H=1040:00 +CPU= 44.849
  16:36:17 STEP 1041 H=1041:00 +CPU= 67.896
  16:37:05 STEP 1042 H=1042:00 +CPU= 44.936
  16:37:51 STEP 1043 H=1043:00 +CPU= 43.715
  16:39:13 STEP 1044 H=1044:00 +CPU= 76.496
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001044
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001044
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001044
  16:40:00 STEP 1045 H=1045:00 +CPU= 43.419
  16:40:47 STEP 1046 H=1046:00 +CPU= 44.521
  16:42:00 STEP 1047 H=1047:00 +CPU= 68.481
  16:42:48 STEP 1048 H=1048:00 +CPU= 44.739
  16:43:35 STEP 1049 H=1049:00 +CPU= 43.683
  16:44:49 STEP 1050 H=1050:00 +CPU= 69.550
  16:45:35 STEP 1051 H=1051:00 +CPU= 43.427
  16:46:23 STEP 1052 H=1052:00 +CPU= 44.487
  16:47:36 STEP 1053 H=1053:00 +CPU= 68.390
  16:48:24 STEP 1054 H=1054:00 +CPU= 44.383
  16:49:11 STEP 1055 H=1055:00 +CPU= 44.862
  16:50:37 STEP 1056 H=1056:00 +CPU= 80.141
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001056
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001056
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001032
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001044
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001044
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001044
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_43.zip
Uploading the intermediate file: upload_file_43.zip
  16:51:23 STEP 1057 H=1057:00 +CPU= 43.369
  16:52:11 STEP 1058 H=1058:00 +CPU= 44.610
  16:53:24 STEP 1059 H=1059:00 +CPU= 68.237
  16:54:11 STEP 1060 H=1060:00 +CPU= 44.329
  16:54:57 STEP 1061 H=1061:00 +CPU= 43.406
  16:56:11 STEP 1062 H=1062:00 +CPU= 69.312
  16:56:58 STEP 1063 H=1063:00 +CPU= 43.375
  16:57:45 STEP 1064 H=1064:00 +CPU= 44.746
  16:58:58 STEP 1065 H=1065:00 +CPU= 68.188
  16:59:46 STEP 1066 H=1066:00 +CPU= 44.717
  17:00:32 STEP 1067 H=1067:00 +CPU= 43.154
  17:01:54 STEP 1068 H=1068:00 +CPU= 76.546
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001068
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001068
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001068
  17:02:40 STEP 1069 H=1069:00 +CPU= 43.383
  17:03:28 STEP 1070 H=1070:00 +CPU= 44.852
  17:04:41 STEP 1071 H=1071:00 +CPU= 68.241
  17:05:29 STEP 1072 H=1072:00 +CPU= 44.699
  17:06:15 STEP 1073 H=1073:00 +CPU= 43.129
  17:07:30 STEP 1074 H=1074:00 +CPU= 70.036
  17:08:16 STEP 1075 H=1075:00 +CPU= 43.299
  17:09:04 STEP 1076 H=1076:00 +CPU= 44.729
  17:10:17 STEP 1077 H=1077:00 +CPU= 68.358
  17:11:04 STEP 1078 H=1078:00 +CPU= 44.635
  17:11:52 STEP 1079 H=1079:00 +CPU= 44.654
  17:13:17 STEP 1080 H=1080:00 +CPU= 80.181
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001080
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001080
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001056
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001068
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001068
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001068
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_44.zip
Uploading the intermediate file: upload_file_44.zip
Uploading trickle at timestep: 3888000
  17:14:04 STEP 1081 H=1081:00 +CPU= 43.120
  17:14:51 STEP 1082 H=1082:00 +CPU= 44.761
  17:16:05 STEP 1083 H=1083:00 +CPU= 68.478
  17:16:52 STEP 1084 H=1084:00 +CPU= 44.804
  17:17:39 STEP 1085 H=1085:00 +CPU= 43.612
  17:18:54 STEP 1086 H=1086:00 +CPU= 69.819
  17:19:40 STEP 1087 H=1087:00 +CPU= 43.741
  17:20:28 STEP 1088 H=1088:00 +CPU= 44.672
  17:21:41 STEP 1089 H=1089:00 +CPU= 68.754
  17:22:29 STEP 1090 H=1090:00 +CPU= 44.935
  17:23:16 STEP 1091 H=1091:00 +CPU= 43.669
  17:24:38 STEP 1092 H=1092:00 +CPU= 76.618
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001092
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001092
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001092
  17:25:24 STEP 1093 H=1093:00 +CPU= 43.748
  17:26:12 STEP 1094 H=1094:00 +CPU= 44.826
  17:27:25 STEP 1095 H=1095:00 +CPU= 68.672
  17:28:13 STEP 1096 H=1096:00 +CPU= 44.606
  17:28:59 STEP 1097 H=1097:00 +CPU= 43.552
  17:30:14 STEP 1098 H=1098:00 +CPU= 70.075
  17:31:00 STEP 1099 H=1099:00 +CPU= 43.495
  17:31:48 STEP 1100 H=1100:00 +CPU= 44.661
  17:33:01 STEP 1101 H=1101:00 +CPU= 68.658
  17:33:49 STEP 1102 H=1102:00 +CPU= 44.727
  17:34:37 STEP 1103 H=1103:00 +CPU= 44.689
  17:36:04 STEP 1104 H=1104:00 +CPU= 80.103
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001104
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001104
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001080
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001092
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001092
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001092
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_45.zip
Uploading the intermediate file: upload_file_45.zip
  17:36:50 STEP 1105 H=1105:00 +CPU= 43.316
  17:37:37 STEP 1106 H=1106:00 +CPU= 44.710
  17:38:51 STEP 1107 H=1107:00 +CPU= 68.478
  17:39:39 STEP 1108 H=1108:00 +CPU= 44.785
  17:40:25 STEP 1109 H=1109:00 +CPU= 43.114
  17:41:39 STEP 1110 H=1110:00 +CPU= 70.066
  17:42:26 STEP 1111 H=1111:00 +CPU= 43.422
  17:43:13 STEP 1112 H=1112:00 +CPU= 44.781
  17:44:26 STEP 1113 H=1113:00 +CPU= 68.284
  17:45:14 STEP 1114 H=1114:00 +CPU= 44.744
  17:46:01 STEP 1115 H=1115:00 +CPU= 43.551
  17:47:22 STEP 1116 H=1116:00 +CPU= 76.673
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001116
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001116
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001116
  17:48:09 STEP 1117 H=1117:00 +CPU= 43.306
  17:48:57 STEP 1118 H=1118:00 +CPU= 45.030
  17:50:10 STEP 1119 H=1119:00 +CPU= 68.623
  17:50:58 STEP 1120 H=1120:00 +CPU= 44.932
  17:51:45 STEP 1121 H=1121:00 +CPU= 43.432
  17:52:59 STEP 1122 H=1122:00 +CPU= 69.684
  17:53:45 STEP 1123 H=1123:00 +CPU= 43.438
  17:54:33 STEP 1124 H=1124:00 +CPU= 44.523
  17:55:46 STEP 1125 H=1125:00 +CPU= 68.557
  17:56:34 STEP 1126 H=1126:00 +CPU= 44.338
  17:57:22 STEP 1127 H=1127:00 +CPU= 44.975
  17:58:48 STEP 1128 H=1128:00 +CPU= 80.164
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001128
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001128
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001104
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001116
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001116
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001116
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_46.zip
Uploading the intermediate file: upload_file_46.zip
  17:59:34 STEP 1129 H=1129:00 +CPU= 43.364
  18:00:22 STEP 1130 H=1130:00 +CPU= 44.792
  18:01:35 STEP 1131 H=1131:00 +CPU= 68.570
  18:02:23 STEP 1132 H=1132:00 +CPU= 44.812
  18:03:09 STEP 1133 H=1133:00 +CPU= 43.278
  18:04:23 STEP 1134 H=1134:00 +CPU= 69.775
  18:05:10 STEP 1135 H=1135:00 +CPU= 43.623
  18:05:58 STEP 1136 H=1136:00 +CPU= 44.822
  18:07:12 STEP 1137 H=1137:00 +CPU= 68.641
  18:07:59 STEP 1138 H=1138:00 +CPU= 44.749
  18:08:46 STEP 1139 H=1139:00 +CPU= 43.692
  18:10:08 STEP 1140 H=1140:00 +CPU= 76.409
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001140
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001140
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001140
  18:10:54 STEP 1141 H=1141:00 +CPU= 43.568
  18:11:42 STEP 1142 H=1142:00 +CPU= 44.786
  18:12:54 STEP 1143 H=1143:00 +CPU= 68.179
  18:13:43 STEP 1144 H=1144:00 +CPU= 44.984
  18:14:29 STEP 1145 H=1145:00 +CPU= 43.364
  18:15:43 STEP 1146 H=1146:00 +CPU= 69.833
  18:16:30 STEP 1147 H=1147:00 +CPU= 43.239
  18:17:17 STEP 1148 H=1148:00 +CPU= 44.773
  18:18:30 STEP 1149 H=1149:00 +CPU= 68.370
  18:19:18 STEP 1150 H=1150:00 +CPU= 44.616
  18:20:05 STEP 1151 H=1151:00 +CPU= 44.732
  18:21:31 STEP 1152 H=1152:00 +CPU= 80.204
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001152
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001152
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001128
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001140
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001140
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001140
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_47.zip
Uploading the intermediate file: upload_file_47.zip
  18:22:17 STEP 1153 H=1153:00 +CPU= 43.246
  18:23:05 STEP 1154 H=1154:00 +CPU= 44.746
  18:24:18 STEP 1155 H=1155:00 +CPU= 68.518
  18:25:06 STEP 1156 H=1156:00 +CPU= 44.735
  18:25:52 STEP 1157 H=1157:00 +CPU= 43.531
  18:27:07 STEP 1158 H=1158:00 +CPU= 70.023
  18:27:53 STEP 1159 H=1159:00 +CPU= 43.318
  18:28:40 STEP 1160 H=1160:00 +CPU= 44.861
  18:29:53 STEP 1161 H=1161:00 +CPU= 68.470
  18:30:41 STEP 1162 H=1162:00 +CPU= 44.818
  18:31:28 STEP 1163 H=1163:00 +CPU= 43.577
  18:32:49 STEP 1164 H=1164:00 +CPU= 76.581
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001164
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001164
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001164
  18:33:35 STEP 1165 H=1165:00 +CPU= 43.431
  18:34:23 STEP 1166 H=1166:00 +CPU= 44.626
  18:35:36 STEP 1167 H=1167:00 +CPU= 68.436
  18:36:24 STEP 1168 H=1168:00 +CPU= 44.813
  18:37:10 STEP 1169 H=1169:00 +CPU= 43.383
  18:38:24 STEP 1170 H=1170:00 +CPU= 69.803
  18:39:11 STEP 1171 H=1171:00 +CPU= 43.397
  18:39:58 STEP 1172 H=1172:00 +CPU= 44.716
  18:41:11 STEP 1173 H=1173:00 +CPU= 68.298
  18:41:59 STEP 1174 H=1174:00 +CPU= 44.700
  18:42:46 STEP 1175 H=1175:00 +CPU= 44.852
  18:44:12 STEP 1176 H=1176:00 +CPU= 80.252
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001176
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001176
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001152
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001164
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001164
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001164
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_48.zip
Uploading the intermediate file: upload_file_48.zip
  18:44:58 STEP 1177 H=1177:00 +CPU= 43.256
  18:45:46 STEP 1178 H=1178:00 +CPU= 44.835
  18:46:59 STEP 1179 H=1179:00 +CPU= 68.165
  18:47:46 STEP 1180 H=1180:00 +CPU= 44.711
  18:48:33 STEP 1181 H=1181:00 +CPU= 43.431
  18:49:47 STEP 1182 H=1182:00 +CPU= 69.687
  18:50:33 STEP 1183 H=1183:00 +CPU= 43.521
  18:51:21 STEP 1184 H=1184:00 +CPU= 44.817
  18:52:34 STEP 1185 H=1185:00 +CPU= 68.510
  18:53:22 STEP 1186 H=1186:00 +CPU= 44.863
  18:54:08 STEP 1187 H=1187:00 +CPU= 43.277
  18:55:30 STEP 1188 H=1188:00 +CPU= 76.540
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001188
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001188
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001188
  18:56:16 STEP 1189 H=1189:00 +CPU= 43.586
  18:57:04 STEP 1190 H=1190:00 +CPU= 44.987
  18:58:17 STEP 1191 H=1191:00 +CPU= 68.488
  18:59:05 STEP 1192 H=1192:00 +CPU= 44.743
  18:59:51 STEP 1193 H=1193:00 +CPU= 43.441
  19:01:06 STEP 1194 H=1194:00 +CPU= 69.751
  19:01:52 STEP 1195 H=1195:00 +CPU= 43.339
  19:02:39 STEP 1196 H=1196:00 +CPU= 44.681
  19:03:52 STEP 1197 H=1197:00 +CPU= 68.383
  19:04:40 STEP 1198 H=1198:00 +CPU= 44.639
  19:05:27 STEP 1199 H=1199:00 +CPU= 44.811
  19:06:54 STEP 1200 H=1200:00 +CPU= 80.495
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001200
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001200
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001176
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001188
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001188
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001188
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_49.zip
Uploading the intermediate file: upload_file_49.zip
  19:07:40 STEP 1201 H=1201:00 +CPU= 43.329
  19:08:27 STEP 1202 H=1202:00 +CPU= 44.642
  19:09:41 STEP 1203 H=1203:00 +CPU= 68.513
  19:10:28 STEP 1204 H=1204:00 +CPU= 44.656
  19:11:14 STEP 1205 H=1205:00 +CPU= 43.387
  19:12:29 STEP 1206 H=1206:00 +CPU= 69.915
  19:13:16 STEP 1207 H=1207:00 +CPU= 43.515
  19:14:03 STEP 1208 H=1208:00 +CPU= 44.709
  19:15:16 STEP 1209 H=1209:00 +CPU= 68.366
  19:16:04 STEP 1210 H=1210:00 +CPU= 44.673
  19:16:50 STEP 1211 H=1211:00 +CPU= 43.504
  19:18:11 STEP 1212 H=1212:00 +CPU= 76.374
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001212
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001212
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001212
  19:18:58 STEP 1213 H=1213:00 +CPU= 43.565
  19:19:46 STEP 1214 H=1214:00 +CPU= 44.794
  19:20:59 STEP 1215 H=1215:00 +CPU= 68.620
  19:21:47 STEP 1216 H=1216:00 +CPU= 44.632
  19:22:33 STEP 1217 H=1217:00 +CPU= 43.307
  19:23:47 STEP 1218 H=1218:00 +CPU= 69.823
  19:24:33 STEP 1219 H=1219:00 +CPU= 43.170
  19:25:21 STEP 1220 H=1220:00 +CPU= 44.643
  19:26:34 STEP 1221 H=1221:00 +CPU= 68.175
  19:27:21 STEP 1222 H=1222:00 +CPU= 44.523
  19:28:09 STEP 1223 H=1223:00 +CPU= 44.754
  19:29:35 STEP 1224 H=1224:00 +CPU= 80.347
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001224
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001224
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001200
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001212
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001212
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001212
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_50.zip
Uploading the intermediate file: upload_file_50.zip
  19:30:21 STEP 1225 H=1225:00 +CPU= 43.361
  19:31:09 STEP 1226 H=1226:00 +CPU= 44.674
  19:32:22 STEP 1227 H=1227:00 +CPU= 68.218
  19:33:10 STEP 1228 H=1228:00 +CPU= 44.835
  19:33:56 STEP 1229 H=1229:00 +CPU= 43.489
  19:35:10 STEP 1230 H=1230:00 +CPU= 69.941
  19:35:56 STEP 1231 H=1231:00 +CPU= 43.283
  19:36:44 STEP 1232 H=1232:00 +CPU= 44.782
  19:37:57 STEP 1233 H=1233:00 +CPU= 68.499
  19:38:45 STEP 1234 H=1234:00 +CPU= 44.790
  19:39:31 STEP 1235 H=1235:00 +CPU= 43.352
  19:40:53 STEP 1236 H=1236:00 +CPU= 76.631
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001236
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001236
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001236
  19:41:39 STEP 1237 H=1237:00 +CPU= 43.625
  19:42:27 STEP 1238 H=1238:00 +CPU= 44.899
  19:43:40 STEP 1239 H=1239:00 +CPU= 68.523
  19:44:28 STEP 1240 H=1240:00 +CPU= 44.539
  19:45:14 STEP 1241 H=1241:00 +CPU= 43.499
  19:46:29 STEP 1242 H=1242:00 +CPU= 70.092
  19:47:15 STEP 1243 H=1243:00 +CPU= 43.426
  19:48:03 STEP 1244 H=1244:00 +CPU= 44.643
  19:49:16 STEP 1245 H=1245:00 +CPU= 68.439
  19:50:04 STEP 1246 H=1246:00 +CPU= 44.722
  19:50:51 STEP 1247 H=1247:00 +CPU= 44.664
  19:52:17 STEP 1248 H=1248:00 +CPU= 79.987
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001248
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001248
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001224
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001236
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001236
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001236
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_51.zip
Uploading the intermediate file: upload_file_51.zip
  19:53:03 STEP 1249 H=1249:00 +CPU= 43.501
  19:53:51 STEP 1250 H=1250:00 +CPU= 44.779
  19:55:04 STEP 1251 H=1251:00 +CPU= 68.188
  19:55:51 STEP 1252 H=1252:00 +CPU= 44.686
  19:56:38 STEP 1253 H=1253:00 +CPU= 43.537
  19:57:53 STEP 1254 H=1254:00 +CPU= 70.010
  19:58:39 STEP 1255 H=1255:00 +CPU= 43.606
  19:59:27 STEP 1256 H=1256:00 +CPU= 44.748
  20:00:40 STEP 1257 H=1257:00 +CPU= 68.580
  20:01:28 STEP 1258 H=1258:00 +CPU= 44.831
  20:02:14 STEP 1259 H=1259:00 +CPU= 43.543
  20:03:36 STEP 1260 H=1260:00 +CPU= 76.653
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001260
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001260
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001260
  20:04:23 STEP 1261 H=1261:00 +CPU= 43.571
  20:05:10 STEP 1262 H=1262:00 +CPU= 44.838
  20:06:24 STEP 1263 H=1263:00 +CPU= 68.681
  20:07:12 STEP 1264 H=1264:00 +CPU= 44.837
  20:07:58 STEP 1265 H=1265:00 +CPU= 43.377
  20:09:12 STEP 1266 H=1266:00 +CPU= 69.932
  20:09:59 STEP 1267 H=1267:00 +CPU= 43.501
  20:10:47 STEP 1268 H=1268:00 +CPU= 44.815
  20:12:00 STEP 1269 H=1269:00 +CPU= 68.571
  20:12:48 STEP 1270 H=1270:00 +CPU= 44.884
  20:13:35 STEP 1271 H=1271:00 +CPU= 44.766
  20:15:02 STEP 1272 H=1272:00 +CPU= 80.489
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001272
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001272
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001248
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001260
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001260
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001260
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_52.zip
Uploading the intermediate file: upload_file_52.zip
  20:15:48 STEP 1273 H=1273:00 +CPU= 43.434
  20:16:36 STEP 1274 H=1274:00 +CPU= 44.769
  20:17:49 STEP 1275 H=1275:00 +CPU= 68.523
  20:18:36 STEP 1276 H=1276:00 +CPU= 44.730
  20:19:23 STEP 1277 H=1277:00 +CPU= 43.530
  20:20:37 STEP 1278 H=1278:00 +CPU= 69.843
  20:21:24 STEP 1279 H=1279:00 +CPU= 43.655
  20:22:11 STEP 1280 H=1280:00 +CPU= 44.820
  20:23:24 STEP 1281 H=1281:00 +CPU= 68.260
  20:24:12 STEP 1282 H=1282:00 +CPU= 44.818
  20:24:58 STEP 1283 H=1283:00 +CPU= 43.569
  20:26:20 STEP 1284 H=1284:00 +CPU= 76.334
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001284
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001284
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001284
  20:27:06 STEP 1285 H=1285:00 +CPU= 43.455
  20:27:54 STEP 1286 H=1286:00 +CPU= 44.721
  20:29:07 STEP 1287 H=1287:00 +CPU= 68.519
  20:29:54 STEP 1288 H=1288:00 +CPU= 44.732
  20:30:41 STEP 1289 H=1289:00 +CPU= 43.360
  20:31:55 STEP 1290 H=1290:00 +CPU= 69.722
  20:32:41 STEP 1291 H=1291:00 +CPU= 43.572
  20:33:29 STEP 1292 H=1292:00 +CPU= 44.732
  20:34:42 STEP 1293 H=1293:00 +CPU= 68.426
  20:35:30 STEP 1294 H=1294:00 +CPU= 44.581
  20:36:17 STEP 1295 H=1295:00 +CPU= 44.820
  20:37:43 STEP 1296 H=1296:00 +CPU= 80.156
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001296
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001296
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001272
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001284
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001284
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001284
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_53.zip
Uploading the intermediate file: upload_file_53.zip
  20:38:29 STEP 1297 H=1297:00 +CPU= 43.373
  20:39:17 STEP 1298 H=1298:00 +CPU= 44.748
  20:40:29 STEP 1299 H=1299:00 +CPU= 68.161
  20:41:17 STEP 1300 H=1300:00 +CPU= 44.652
  20:42:03 STEP 1301 H=1301:00 +CPU= 43.505
  20:43:18 STEP 1302 H=1302:00 +CPU= 69.716
  20:44:04 STEP 1303 H=1303:00 +CPU= 43.189
  20:44:52 STEP 1304 H=1304:00 +CPU= 44.890
  20:46:05 STEP 1305 H=1305:00 +CPU= 68.311
  20:46:52 STEP 1306 H=1306:00 +CPU= 44.960
  20:47:39 STEP 1307 H=1307:00 +CPU= 43.597
  20:49:00 STEP 1308 H=1308:00 +CPU= 76.299
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001308
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001308
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001308
  20:49:46 STEP 1309 H=1309:00 +CPU= 43.316
  20:50:34 STEP 1310 H=1310:00 +CPU= 44.767
  20:51:47 STEP 1311 H=1311:00 +CPU= 68.407
  20:52:35 STEP 1312 H=1312:00 +CPU= 44.811
  20:53:21 STEP 1313 H=1313:00 +CPU= 43.517
  20:54:35 STEP 1314 H=1314:00 +CPU= 69.617
  20:55:22 STEP 1315 H=1315:00 +CPU= 43.410
  20:56:10 STEP 1316 H=1316:00 +CPU= 44.833
  20:57:22 STEP 1317 H=1317:00 +CPU= 68.221
  20:58:10 STEP 1318 H=1318:00 +CPU= 44.718
  20:58:58 STEP 1319 H=1319:00 +CPU= 44.764
  21:00:24 STEP 1320 H=1320:00 +CPU= 80.306
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001320
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001320
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001296
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001308
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001308
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001308
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_54.zip
Uploading the intermediate file: upload_file_54.zip
Uploading trickle at timestep: 4752000
  21:01:10 STEP 1321 H=1321:00 +CPU= 43.579
  21:01:58 STEP 1322 H=1322:00 +CPU= 44.813
  21:03:11 STEP 1323 H=1323:00 +CPU= 68.397
  21:03:59 STEP 1324 H=1324:00 +CPU= 44.690
  21:04:45 STEP 1325 H=1325:00 +CPU= 43.463
  21:06:00 STEP 1326 H=1326:00 +CPU= 69.874
  21:06:46 STEP 1327 H=1327:00 +CPU= 43.498
  21:07:33 STEP 1328 H=1328:00 +CPU= 44.553
  21:08:46 STEP 1329 H=1329:00 +CPU= 68.353
  21:09:34 STEP 1330 H=1330:00 +CPU= 44.670
  21:10:20 STEP 1331 H=1331:00 +CPU= 43.390
  21:11:42 STEP 1332 H=1332:00 +CPU= 76.707
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001332
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001332
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001332
  21:12:28 STEP 1333 H=1333:00 +CPU= 43.471
  21:13:16 STEP 1334 H=1334:00 +CPU= 44.801
  21:14:29 STEP 1335 H=1335:00 +CPU= 68.472
  21:15:16 STEP 1336 H=1336:00 +CPU= 44.697
  21:16:03 STEP 1337 H=1337:00 +CPU= 43.427
  21:17:17 STEP 1338 H=1338:00 +CPU= 69.908
  21:18:03 STEP 1339 H=1339:00 +CPU= 43.413
  21:18:51 STEP 1340 H=1340:00 +CPU= 44.769
  21:20:04 STEP 1341 H=1341:00 +CPU= 68.160
  21:20:51 STEP 1342 H=1342:00 +CPU= 44.704
  21:21:39 STEP 1343 H=1343:00 +CPU= 44.888
  21:23:05 STEP 1344 H=1344:00 +CPU= 80.104
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001344
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001344
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001320
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001332
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001332
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001332
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_55.zip
Uploading the intermediate file: upload_file_55.zip
  21:23:52 STEP 1345 H=1345:00 +CPU= 43.399
  21:24:39 STEP 1346 H=1346:00 +CPU= 44.745
  21:25:52 STEP 1347 H=1347:00 +CPU= 68.117
  21:26:39 STEP 1348 H=1348:00 +CPU= 44.670
  21:27:26 STEP 1349 H=1349:00 +CPU= 43.559
  21:28:40 STEP 1350 H=1350:00 +CPU= 69.763
  21:29:26 STEP 1351 H=1351:00 +CPU= 43.533
  21:30:14 STEP 1352 H=1352:00 +CPU= 44.742
  21:31:27 STEP 1353 H=1353:00 +CPU= 68.438
  21:32:15 STEP 1354 H=1354:00 +CPU= 44.887
  21:33:01 STEP 1355 H=1355:00 +CPU= 43.386
  21:34:23 STEP 1356 H=1356:00 +CPU= 76.574
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001356
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001356
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001356
  21:35:09 STEP 1357 H=1357:00 +CPU= 43.474
  21:35:56 STEP 1358 H=1358:00 +CPU= 44.703
  21:37:09 STEP 1359 H=1359:00 +CPU= 68.180
  21:37:57 STEP 1360 H=1360:00 +CPU= 44.703
  21:38:43 STEP 1361 H=1361:00 +CPU= 43.532
  21:39:57 STEP 1362 H=1362:00 +CPU= 69.765
  21:40:43 STEP 1363 H=1363:00 +CPU= 43.528
  21:41:31 STEP 1364 H=1364:00 +CPU= 44.716
  21:42:44 STEP 1365 H=1365:00 +CPU= 68.530
  21:43:32 STEP 1366 H=1366:00 +CPU= 44.616
  21:44:19 STEP 1367 H=1367:00 +CPU= 44.666
  21:45:45 STEP 1368 H=1368:00 +CPU= 80.304
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001368
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001368
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001344
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001356
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001356
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001356
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_56.zip
Uploading the intermediate file: upload_file_56.zip
  21:46:31 STEP 1369 H=1369:00 +CPU= 43.372
  21:47:19 STEP 1370 H=1370:00 +CPU= 44.620
  21:48:32 STEP 1371 H=1371:00 +CPU= 68.292
  21:49:19 STEP 1372 H=1372:00 +CPU= 44.682
  21:50:05 STEP 1373 H=1373:00 +CPU= 43.428
  21:51:19 STEP 1374 H=1374:00 +CPU= 69.708
  21:52:06 STEP 1375 H=1375:00 +CPU= 43.453
  21:52:53 STEP 1376 H=1376:00 +CPU= 44.786
  21:54:06 STEP 1377 H=1377:00 +CPU= 68.521
  21:54:54 STEP 1378 H=1378:00 +CPU= 44.780
  21:55:40 STEP 1379 H=1379:00 +CPU= 43.644
  21:57:02 STEP 1380 H=1380:00 +CPU= 76.640
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001380
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001380
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001380
  21:57:48 STEP 1381 H=1381:00 +CPU= 43.437
  21:58:36 STEP 1382 H=1382:00 +CPU= 44.697
  21:59:49 STEP 1383 H=1383:00 +CPU= 68.353
  22:00:36 STEP 1384 H=1384:00 +CPU= 44.794
  22:01:22 STEP 1385 H=1385:00 +CPU= 43.430
  22:02:37 STEP 1386 H=1386:00 +CPU= 69.784
  22:03:23 STEP 1387 H=1387:00 +CPU= 43.431
  22:04:11 STEP 1388 H=1388:00 +CPU= 44.629
  22:05:23 STEP 1389 H=1389:00 +CPU= 68.347
  22:06:11 STEP 1390 H=1390:00 +CPU= 44.681
  22:06:59 STEP 1391 H=1391:00 +CPU= 44.757
  22:08:24 STEP 1392 H=1392:00 +CPU= 80.328
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001392
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001392
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001368
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001380
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001380
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001380
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_57.zip
Uploading the intermediate file: upload_file_57.zip
  22:09:11 STEP 1393 H=1393:00 +CPU= 43.365
  22:09:58 STEP 1394 H=1394:00 +CPU= 44.774
  22:11:11 STEP 1395 H=1395:00 +CPU= 68.396
  22:11:59 STEP 1396 H=1396:00 +CPU= 44.619
  22:12:45 STEP 1397 H=1397:00 +CPU= 43.376
  22:13:59 STEP 1398 H=1398:00 +CPU= 69.908
  22:14:46 STEP 1399 H=1399:00 +CPU= 43.484
  22:15:33 STEP 1400 H=1400:00 +CPU= 44.746
  22:16:46 STEP 1401 H=1401:00 +CPU= 68.528
  22:17:34 STEP 1402 H=1402:00 +CPU= 44.805
  22:18:20 STEP 1403 H=1403:00 +CPU= 43.505
  22:19:42 STEP 1404 H=1404:00 +CPU= 76.582
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001404
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001404
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001404
  22:20:28 STEP 1405 H=1405:00 +CPU= 43.577
  22:21:16 STEP 1406 H=1406:00 +CPU= 44.828
  22:22:29 STEP 1407 H=1407:00 +CPU= 68.383
  22:23:17 STEP 1408 H=1408:00 +CPU= 44.738
  22:24:03 STEP 1409 H=1409:00 +CPU= 43.415
  22:25:17 STEP 1410 H=1410:00 +CPU= 69.919
  22:26:04 STEP 1411 H=1411:00 +CPU= 43.411
  22:26:51 STEP 1412 H=1412:00 +CPU= 44.743
  22:28:04 STEP 1413 H=1413:00 +CPU= 68.276
  22:28:52 STEP 1414 H=1414:00 +CPU= 44.742
  22:29:39 STEP 1415 H=1415:00 +CPU= 44.836
  22:31:05 STEP 1416 H=1416:00 +CPU= 80.276
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001416
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001416
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001392
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001404
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001404
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001404
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_58.zip
Uploading the intermediate file: upload_file_58.zip
  22:31:52 STEP 1417 H=1417:00 +CPU= 43.397
  22:32:39 STEP 1418 H=1418:00 +CPU= 44.442
  22:33:52 STEP 1419 H=1419:00 +CPU= 68.655
  22:34:40 STEP 1420 H=1420:00 +CPU= 44.803
  22:35:27 STEP 1421 H=1421:00 +CPU= 43.579
  22:36:41 STEP 1422 H=1422:00 +CPU= 69.771
  22:37:27 STEP 1423 H=1423:00 +CPU= 43.686
  22:38:15 STEP 1424 H=1424:00 +CPU= 44.819
  22:39:28 STEP 1425 H=1425:00 +CPU= 68.194
  22:40:15 STEP 1426 H=1426:00 +CPU= 44.694
  22:41:02 STEP 1427 H=1427:00 +CPU= 43.446
  22:42:23 STEP 1428 H=1428:00 +CPU= 76.620
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001428
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001428
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001428
  22:43:10 STEP 1429 H=1429:00 +CPU= 43.548
  22:43:57 STEP 1430 H=1430:00 +CPU= 44.910
  22:45:10 STEP 1431 H=1431:00 +CPU= 68.517
  22:45:58 STEP 1432 H=1432:00 +CPU= 44.702
  22:46:45 STEP 1433 H=1433:00 +CPU= 43.497
  22:47:59 STEP 1434 H=1434:00 +CPU= 69.941
  22:48:45 STEP 1435 H=1435:00 +CPU= 43.390
  22:49:33 STEP 1436 H=1436:00 +CPU= 44.863
  22:50:46 STEP 1437 H=1437:00 +CPU= 68.289
  22:51:33 STEP 1438 H=1438:00 +CPU= 44.688
  22:52:21 STEP 1439 H=1439:00 +CPU= 44.835
  22:53:46 STEP 1440 H=1440:00 +CPU= 80.149
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001440
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001440
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001440
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001416
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001428
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001428
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001428
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_59.zip
Uploading the intermediate file: upload_file_59.zip
  22:54:33 STEP 1441 H=1441:00 +CPU= 43.542
  22:55:21 STEP 1442 H=1442:00 +CPU= 44.617
  22:56:33 STEP 1443 H=1443:00 +CPU= 68.249
  22:57:21 STEP 1444 H=1444:00 +CPU= 44.686
  22:58:07 STEP 1445 H=1445:00 +CPU= 43.568
  22:59:22 STEP 1446 H=1446:00 +CPU= 69.905
  23:00:08 STEP 1447 H=1447:00 +CPU= 43.493
  23:00:55 STEP 1448 H=1448:00 +CPU= 44.724
  23:02:08 STEP 1449 H=1449:00 +CPU= 68.455
  23:02:56 STEP 1450 H=1450:00 +CPU= 44.899
  23:03:42 STEP 1451 H=1451:00 +CPU= 43.447
  23:05:04 STEP 1452 H=1452:00 +CPU= 76.465
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001452
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001452
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001452
  23:05:50 STEP 1453 H=1453:00 +CPU= 43.427
  23:06:38 STEP 1454 H=1454:00 +CPU= 44.725
  23:07:50 STEP 1455 H=1455:00 +CPU= 68.325
  23:08:38 STEP 1456 H=1456:00 +CPU= 44.629
  23:09:24 STEP 1457 H=1457:00 +CPU= 43.416
  23:10:38 STEP 1458 H=1458:00 +CPU= 69.775
  23:11:24 STEP 1459 H=1459:00 +CPU= 43.195
  23:12:12 STEP 1460 H=1460:00 +CPU= 44.854
  23:13:25 STEP 1461 H=1461:00 +CPU= 68.414
  23:14:13 STEP 1462 H=1462:00 +CPU= 44.866
  23:15:01 STEP 1463 H=1463:00 +CPU= 44.834
  23:16:27 STEP 1464 H=1464:00 +CPU= 80.311
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001464
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001464
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001464
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001440
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001440
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001440
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001452
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001452
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001452
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_60.zip
Uploading the intermediate file: upload_file_60.zip
  23:17:13 STEP 1465 H=1465:00 +CPU= 43.334
  23:18:01 STEP 1466 H=1466:00 +CPU= 44.869
  23:19:13 STEP 1467 H=1467:00 +CPU= 68.246
  23:20:01 STEP 1468 H=1468:00 +CPU= 44.662
  23:20:48 STEP 1469 H=1469:00 +CPU= 43.646
  23:22:02 STEP 1470 H=1470:00 +CPU= 69.648
  23:22:48 STEP 1471 H=1471:00 +CPU= 43.449
  23:23:36 STEP 1472 H=1472:00 +CPU= 44.976
  23:24:49 STEP 1473 H=1473:00 +CPU= 68.264
  23:25:37 STEP 1474 H=1474:00 +CPU= 45.011
  23:26:23 STEP 1475 H=1475:00 +CPU= 43.454
  23:27:45 STEP 1476 H=1476:00 +CPU= 76.471
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001476
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001476
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001476
  23:28:31 STEP 1477 H=1477:00 +CPU= 43.636
  23:29:19 STEP 1478 H=1478:00 +CPU= 44.856
  23:30:32 STEP 1479 H=1479:00 +CPU= 68.333
  23:31:19 STEP 1480 H=1480:00 +CPU= 44.813
  23:32:06 STEP 1481 H=1481:00 +CPU= 43.446
  23:33:20 STEP 1482 H=1482:00 +CPU= 69.937
  23:34:07 STEP 1483 H=1483:00 +CPU= 43.485
  23:34:54 STEP 1484 H=1484:00 +CPU= 44.905
  23:36:07 STEP 1485 H=1485:00 +CPU= 68.356
  23:36:55 STEP 1486 H=1486:00 +CPU= 44.898
  23:37:43 STEP 1487 H=1487:00 +CPU= 44.887
  23:39:08 STEP 1488 H=1488:00 +CPU= 80.266
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001488
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001488
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001488
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001464
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001464
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001464
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001476
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001476
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001476
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_61.zip
Uploading the intermediate file: upload_file_61.zip
  23:39:55 STEP 1489 H=1489:00 +CPU= 43.301
  23:40:42 STEP 1490 H=1490:00 +CPU= 44.731
  23:41:55 STEP 1491 H=1491:00 +CPU= 68.346
  23:42:43 STEP 1492 H=1492:00 +CPU= 44.683
  23:43:29 STEP 1493 H=1493:00 +CPU= 43.666
  23:44:43 STEP 1494 H=1494:00 +CPU= 69.869
  23:45:30 STEP 1495 H=1495:00 +CPU= 43.629
  23:46:18 STEP 1496 H=1496:00 +CPU= 44.649
  23:47:30 STEP 1497 H=1497:00 +CPU= 68.389
  23:48:18 STEP 1498 H=1498:00 +CPU= 44.874
  23:49:05 STEP 1499 H=1499:00 +CPU= 43.562
  23:50:26 STEP 1500 H=1500:00 +CPU= 76.437
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001500
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001500
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001500
  23:51:12 STEP 1501 H=1501:00 +CPU= 43.555
  23:52:00 STEP 1502 H=1502:00 +CPU= 44.762
  23:53:13 STEP 1503 H=1503:00 +CPU= 68.499
  23:54:01 STEP 1504 H=1504:00 +CPU= 44.834
  23:54:47 STEP 1505 H=1505:00 +CPU= 43.480
  23:56:02 STEP 1506 H=1506:00 +CPU= 69.932
  23:56:48 STEP 1507 H=1507:00 +CPU= 43.514
  23:57:36 STEP 1508 H=1508:00 +CPU= 45.019
  23:58:49 STEP 1509 H=1509:00 +CPU= 68.236
  23:59:36 STEP 1510 H=1510:00 +CPU= 44.954
  00:00:25 STEP 1511 H=1511:00 +CPU= 44.899
  00:01:51 STEP 1512 H=1512:00 +CPU= 80.583
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001512
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001512
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001512
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001488
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001488
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001488
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001500
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001500
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001500
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_62.zip
Uploading the intermediate file: upload_file_62.zip
  00:02:37 STEP 1513 H=1513:00 +CPU= 43.393
  00:03:25 STEP 1514 H=1514:00 +CPU= 44.853
  00:04:38 STEP 1515 H=1515:00 +CPU= 68.497
  00:05:26 STEP 1516 H=1516:00 +CPU= 44.807
  00:06:12 STEP 1517 H=1517:00 +CPU= 43.459
  00:07:26 STEP 1518 H=1518:00 +CPU= 69.781
  00:08:12 STEP 1519 H=1519:00 +CPU= 43.485
  00:09:00 STEP 1520 H=1520:00 +CPU= 44.917
  00:10:13 STEP 1521 H=1521:00 +CPU= 68.423
  00:11:01 STEP 1522 H=1522:00 +CPU= 44.903
  00:11:48 STEP 1523 H=1523:00 +CPU= 43.663
  00:13:09 STEP 1524 H=1524:00 +CPU= 76.697
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001524
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001524
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001524
  00:13:56 STEP 1525 H=1525:00 +CPU= 43.481
  00:14:44 STEP 1526 H=1526:00 +CPU= 44.993
  00:15:57 STEP 1527 H=1527:00 +CPU= 68.759
  00:16:45 STEP 1528 H=1528:00 +CPU= 44.815
  00:17:31 STEP 1529 H=1529:00 +CPU= 43.499
  00:18:45 STEP 1530 H=1530:00 +CPU= 69.856
  00:19:32 STEP 1531 H=1531:00 +CPU= 43.548
  00:20:20 STEP 1532 H=1532:00 +CPU= 44.866
  00:21:32 STEP 1533 H=1533:00 +CPU= 68.380
  00:22:20 STEP 1534 H=1534:00 +CPU= 44.682
  00:23:08 STEP 1535 H=1535:00 +CPU= 44.976
  00:24:34 STEP 1536 H=1536:00 +CPU= 80.209
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001536
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001536
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001536
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001512
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001512
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001512
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001524
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001524
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001524
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_63.zip
Uploading the intermediate file: upload_file_63.zip
  00:25:20 STEP 1537 H=1537:00 +CPU= 43.369
  00:26:08 STEP 1538 H=1538:00 +CPU= 44.751
  00:27:21 STEP 1539 H=1539:00 +CPU= 68.803
  00:28:09 STEP 1540 H=1540:00 +CPU= 44.746
  00:28:55 STEP 1541 H=1541:00 +CPU= 43.553
  00:30:10 STEP 1542 H=1542:00 +CPU= 69.937
  00:30:56 STEP 1543 H=1543:00 +CPU= 43.553
  00:31:44 STEP 1544 H=1544:00 +CPU= 44.949
  00:32:58 STEP 1545 H=1545:00 +CPU= 68.594
  00:33:45 STEP 1546 H=1546:00 +CPU= 44.919
  00:34:32 STEP 1547 H=1547:00 +CPU= 43.469
  00:35:53 STEP 1548 H=1548:00 +CPU= 76.656
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001548
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001548
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001548
  00:36:39 STEP 1549 H=1549:00 +CPU= 43.408
  00:37:27 STEP 1550 H=1550:00 +CPU= 44.765
  00:38:40 STEP 1551 H=1551:00 +CPU= 68.591
  00:39:28 STEP 1552 H=1552:00 +CPU= 44.795
  00:40:14 STEP 1553 H=1553:00 +CPU= 43.351
  00:41:29 STEP 1554 H=1554:00 +CPU= 70.036
  00:42:15 STEP 1555 H=1555:00 +CPU= 43.317
  00:43:02 STEP 1556 H=1556:00 +CPU= 44.825
  00:44:15 STEP 1557 H=1557:00 +CPU= 68.434
  00:45:03 STEP 1558 H=1558:00 +CPU= 44.601
  00:45:50 STEP 1559 H=1559:00 +CPU= 44.864
  00:47:16 STEP 1560 H=1560:00 +CPU= 80.283
Moving to projects directory: /home/luuser/slots/1/ICMGGhq0f+001560
Moving to projects directory: /home/luuser/slots/1/ICMSHhq0f+001560
Moving to projects directory: /home/luuser/slots/1/ICMUAhq0f+001560
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001536
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001536
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001536
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMGGhq0f+001548
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMSHhq0f+001548
Adding to the zip: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_12201048/ICMUAhq0f+001548
Zipping up the intermediate file: /home/luuser/projects/climateprediction.net/oifs_43r3_ps_0404_2015050100_123_984_12201048_0_r979098569_64.zip
Uploading the intermediate file: upload_file_64.zip
Uploading trickle at timestep: 5616000
  00:48:03 STEP 1561 H=1561:00 +CPU= 43.270
  00:48:51 STEP 1562 H=1562:00 +CPU= 44.774
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:36:29 1537854 22287320 oifs_43r3_ps_0404_2015050100_123_984_12201048_0 5,616,000 83,421 0.0149
09 May 2023 16:36:10 1537854 22287320 oifs_43r3_ps_0404_2015050100_123_984_12201048_0 4,752,000 70,650 0.0149
09 May 2023 16:35:52 1537854 22287320 oifs_43r3_ps_0404_2015050100_123_984_12201048_0 3,888,000 57,878 0.0149
09 May 2023 16:35:31 1537854 22287320 oifs_43r3_ps_0404_2015050100_123_984_12201048_0 3,024,000 45,123 0.0149
09 May 2023 16:34:51 1537854 22287320 oifs_43r3_ps_0404_2015050100_123_984_12201048_0 1,728,000 25,493 0.0148
09 May 2023 16:34:29 1537854 22287320 oifs_43r3_ps_0404_2015050100_123_984_12201048_0 864,000 12,802 0.0148


©2024 cpdn.org