climateprediction.net home page
Task 22265721

Task 22265721

Name oifs_43r3_ps_0091_1995050100_123_964_12180735_0
Workunit 12180735
Created 22 Dec 2022, 7:27:26 UTC
Sent 22 Dec 2022, 9:46:59 UTC
Report deadline 21 Jan 2023, 9:46:59 UTC
Received 25 Dec 2022, 7:35:28 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1537254
Run time 22 hours 46 min 10 sec
CPU time 19 hours 44 min 10 sec
Validate state Invalid
Credit 7,015.68
Device peak FLOPS 3.97 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,224.09 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,682.04 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
TEP  914 H= 914:00 +CPU= 32.528
  10:11:33 STEP  915 H= 915:00 +CPU= 52.434
  10:12:07 STEP  916 H= 916:00 +CPU= 34.143
  10:12:40 STEP  917 H= 917:00 +CPU= 33.282
  10:13:34 STEP  918 H= 918:00 +CPU= 54.137
  10:14:08 STEP  919 H= 919:00 +CPU= 33.238
  10:14:42 STEP  920 H= 920:00 +CPU= 34.196
  10:15:35 STEP  921 H= 921:00 +CPU= 52.518
  10:16:09 STEP  922 H= 922:00 +CPU= 34.218
  10:16:42 STEP  923 H= 923:00 +CPU= 33.171
  10:17:41 STEP  924 H= 924:00 +CPU= 58.644
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000924
  10:18:14 STEP  925 H= 925:00 +CPU= 33.207
  10:18:49 STEP  926 H= 926:00 +CPU= 34.176
  10:19:42 STEP  927 H= 927:00 +CPU= 52.876
  10:20:16 STEP  928 H= 928:00 +CPU= 34.130
  10:20:50 STEP  929 H= 929:00 +CPU= 33.487
  10:21:45 STEP  930 H= 930:00 +CPU= 54.127
  10:22:18 STEP  931 H= 931:00 +CPU= 33.326
  10:22:52 STEP  932 H= 932:00 +CPU= 34.120
  10:23:45 STEP  933 H= 933:00 +CPU= 52.516
  10:24:19 STEP  934 H= 934:00 +CPU= 34.186
  10:24:53 STEP  935 H= 935:00 +CPU= 34.290
  10:25:59 STEP  936 H= 936:00 +CPU= 61.689
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_38.zip
Uploading the intermediate file: upload_file_38.zip
  10:26:32 STEP  937 H= 937:00 +CPU= 33.197
  10:27:07 STEP  938 H= 938:00 +CPU= 34.092
  10:28:00 STEP  939 H= 939:00 +CPU= 53.115
  10:28:34 STEP  940 H= 940:00 +CPU= 34.114
  10:29:07 STEP  941 H= 941:00 +CPU= 33.289
  10:30:01 STEP  942 H= 942:00 +CPU= 53.827
  10:30:35 STEP  943 H= 943:00 +CPU= 33.255
  10:31:10 STEP  944 H= 944:00 +CPU= 34.278
  10:32:03 STEP  945 H= 945:00 +CPU= 52.650
  10:32:37 STEP  946 H= 946:00 +CPU= 34.248
  10:33:10 STEP  947 H= 947:00 +CPU= 33.313
  10:34:09 STEP  948 H= 948:00 +CPU= 58.332
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000948
  10:34:42 STEP  949 H= 949:00 +CPU= 33.331
  10:35:16 STEP  950 H= 950:00 +CPU= 34.090
  10:36:09 STEP  951 H= 951:00 +CPU= 52.944
  10:36:44 STEP  952 H= 952:00 +CPU= 34.191
  10:37:17 STEP  953 H= 953:00 +CPU= 33.421
  10:38:11 STEP  954 H= 954:00 +CPU= 54.188
  10:38:45 STEP  955 H= 955:00 +CPU= 33.402
  10:39:19 STEP  956 H= 956:00 +CPU= 34.363
  10:40:12 STEP  957 H= 957:00 +CPU= 52.659
  10:40:46 STEP  958 H= 958:00 +CPU= 34.239
  10:41:21 STEP  959 H= 959:00 +CPU= 34.520
  10:42:24 STEP  960 H= 960:00 +CPU= 62.044
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_39.zip
Uploading the intermediate file: upload_file_39.zip
  10:42:58 STEP  961 H= 961:00 +CPU= 33.426
  10:43:32 STEP  962 H= 962:00 +CPU= 34.353
  10:44:25 STEP  963 H= 963:00 +CPU= 53.280
  10:45:00 STEP  964 H= 964:00 +CPU= 34.383
  10:45:33 STEP  965 H= 965:00 +CPU= 33.483
  10:46:27 STEP  966 H= 966:00 +CPU= 53.912
  10:47:01 STEP  967 H= 967:00 +CPU= 33.460
  10:47:35 STEP  968 H= 968:00 +CPU= 34.435
  10:48:29 STEP  969 H= 969:00 +CPU= 53.044
  10:49:03 STEP  970 H= 970:00 +CPU= 34.411
  10:49:37 STEP  971 H= 971:00 +CPU= 33.540
  10:50:36 STEP  972 H= 972:00 +CPU= 58.808
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000972
  10:51:09 STEP  973 H= 973:00 +CPU= 33.477
  10:51:44 STEP  974 H= 974:00 +CPU= 34.412
  10:52:37 STEP  975 H= 975:00 +CPU= 53.257
  10:53:11 STEP  976 H= 976:00 +CPU= 34.336
  10:53:45 STEP  977 H= 977:00 +CPU= 33.460
  10:54:40 STEP  978 H= 978:00 +CPU= 54.503
  10:55:21 STEP  979 H= 979:00 +CPU= 36.425
  10:56:06 STEP  980 H= 980:00 +CPU= 37.812
  10:57:05 STEP  981 H= 981:00 +CPU= 55.898
  10:57:45 STEP  982 H= 982:00 +CPU= 36.476
  10:58:24 STEP  983 H= 983:00 +CPU= 35.941
  10:59:48 STEP  984 H= 984:00 +CPU= 74.501
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_40.zip
Uploading the intermediate file: upload_file_40.zip
  11:00:29 STEP  985 H= 985:00 +CPU= 41.260
  11:01:28 STEP  986 H= 986:00 +CPU= 44.481
  11:02:41 STEP  987 H= 987:00 +CPU= 66.130
  11:03:22 STEP  988 H= 988:00 +CPU= 40.440
  11:04:04 STEP  989 H= 989:00 +CPU= 41.649
  11:05:09 STEP  990 H= 990:00 +CPU= 65.459
  11:05:50 STEP  991 H= 991:00 +CPU= 38.940
  11:06:32 STEP  992 H= 992:00 +CPU= 41.347
  11:07:35 STEP  993 H= 993:00 +CPU= 62.278
  11:08:18 STEP  994 H= 994:00 +CPU= 42.766
  11:09:01 STEP  995 H= 995:00 +CPU= 41.304
  11:10:13 STEP  996 H= 996:00 +CPU= 71.117
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000996
  11:11:02 STEP  997 H= 997:00 +CPU= 40.276
  11:11:42 STEP  998 H= 998:00 +CPU= 38.717
  11:12:41 STEP  999 H= 999:00 +CPU= 59.333
  11:13:20 STEP 1000 H=1000:00 +CPU= 38.114
  11:13:57 STEP 1001 H=1001:00 +CPU= 37.262
  11:15:06 STEP 1002 H=1002:00 +CPU= 63.585
  11:15:46 STEP 1003 H=1003:00 +CPU= 38.789
  11:16:31 STEP 1004 H=1004:00 +CPU= 41.312
  11:17:35 STEP 1005 H=1005:00 +CPU= 62.630
  11:18:17 STEP 1006 H=1006:00 +CPU= 41.806
  11:19:00 STEP 1007 H=1007:00 +CPU= 42.090
  11:20:34 STEP 1008 H=1008:00 +CPU= 78.734
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_41.zip
Uploading the intermediate file: upload_file_41.zip
  11:21:18 STEP 1009 H=1009:00 +CPU= 40.587
  11:21:58 STEP 1010 H=1010:00 +CPU= 40.569
  11:23:12 STEP 1011 H=1011:00 +CPU= 65.544
  11:23:56 STEP 1012 H=1012:00 +CPU= 42.349
  11:24:37 STEP 1013 H=1013:00 +CPU= 41.032
  11:25:43 STEP 1014 H=1014:00 +CPU= 65.132
  11:26:21 STEP 1015 H=1015:00 +CPU= 38.319
  11:27:07 STEP 1016 H=1016:00 +CPU= 40.057
  11:28:09 STEP 1017 H=1017:00 +CPU= 61.480
  11:28:47 STEP 1018 H=1018:00 +CPU= 37.551
  11:29:23 STEP 1019 H=1019:00 +CPU= 36.006
  11:30:34 STEP 1020 H=1020:00 +CPU= 64.951
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001020
  11:31:12 STEP 1021 H=1021:00 +CPU= 36.321
  11:31:50 STEP 1022 H=1022:00 +CPU= 37.649
  11:32:52 STEP 1023 H=1023:00 +CPU= 58.138
  11:33:30 STEP 1024 H=1024:00 +CPU= 38.220
  11:34:07 STEP 1025 H=1025:00 +CPU= 36.157
  11:35:20 STEP 1026 H=1026:00 +CPU= 60.585
  11:36:12 STEP 1027 H=1027:00 +CPU= 37.378
  11:36:57 STEP 1028 H=1028:00 +CPU= 37.476
  11:38:29 STEP 1029 H=1029:00 +CPU= 60.887
  11:39:17 STEP 1030 H=1030:00 +CPU= 37.757
  11:39:58 STEP 1031 H=1031:00 +CPU= 37.021
  11:41:21 STEP 1032 H=1032:00 +CPU= 66.404
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_42.zip
Uploading the intermediate file: upload_file_42.zip
  11:42:13 STEP 1033 H=1033:00 +CPU= 36.811
  11:43:00 STEP 1034 H=1034:00 +CPU= 37.669
  11:43:58 STEP 1035 H=1035:00 +CPU= 55.798
  11:44:41 STEP 1036 H=1036:00 +CPU= 37.102
  11:45:24 STEP 1037 H=1037:00 +CPU= 36.499
  11:46:21 STEP 1038 H=1038:00 +CPU= 56.869
  11:47:14 STEP 1039 H=1039:00 +CPU= 37.105
  11:47:56 STEP 1040 H=1040:00 +CPU= 37.001
  11:49:03 STEP 1041 H=1041:00 +CPU= 57.227
  11:49:46 STEP 1042 H=1042:00 +CPU= 37.174
  11:50:26 STEP 1043 H=1043:00 +CPU= 35.886
  11:51:47 STEP 1044 H=1044:00 +CPU= 63.744
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001044
  11:52:45 STEP 1045 H=1045:00 +CPU= 37.758
  11:53:29 STEP 1046 H=1046:00 +CPU= 37.151
  11:54:42 STEP 1047 H=1047:00 +CPU= 58.288
  11:55:51 STEP 1048 H=1048:00 +CPU= 38.837
  11:56:38 STEP 1049 H=1049:00 +CPU= 36.772
  11:57:55 STEP 1050 H=1050:00 +CPU= 59.221
  11:58:45 STEP 1051 H=1051:00 +CPU= 36.774
  11:59:30 STEP 1052 H=1052:00 +CPU= 37.414
  12:00:40 STEP 1053 H=1053:00 +CPU= 57.959
  12:01:23 STEP 1054 H=1054:00 +CPU= 36.893
  12:02:06 STEP 1055 H=1055:00 +CPU= 37.299
  12:03:36 STEP 1056 H=1056:00 +CPU= 68.471
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_43.zip
Uploading the intermediate file: upload_file_43.zip
  12:04:25 STEP 1057 H=1057:00 +CPU= 36.459
  12:05:12 STEP 1058 H=1058:00 +CPU= 37.658
  12:06:13 STEP 1059 H=1059:00 +CPU= 56.134
  12:06:59 STEP 1060 H=1060:00 +CPU= 37.656
  12:07:42 STEP 1061 H=1061:00 +CPU= 36.319
  12:08:39 STEP 1062 H=1062:00 +CPU= 56.731
  12:09:17 STEP 1063 H=1063:00 +CPU= 35.431
  12:10:05 STEP 1064 H=1064:00 +CPU= 37.364
  12:11:18 STEP 1065 H=1065:00 +CPU= 57.908
  12:12:17 STEP 1066 H=1066:00 +CPU= 38.756
  12:13:10 STEP 1067 H=1067:00 +CPU= 37.553
  12:14:23 STEP 1068 H=1068:00 +CPU= 63.814
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001068
  12:15:15 STEP 1069 H=1069:00 +CPU= 37.530
  12:16:13 STEP 1070 H=1070:00 +CPU= 38.180
  12:17:19 STEP 1071 H=1071:00 +CPU= 57.146
  12:18:05 STEP 1072 H=1072:00 +CPU= 37.657
  12:18:47 STEP 1073 H=1073:00 +CPU= 36.322
  12:19:54 STEP 1074 H=1074:00 +CPU= 58.208
  12:20:43 STEP 1075 H=1075:00 +CPU= 37.166
  12:21:27 STEP 1076 H=1076:00 +CPU= 37.030
  12:22:34 STEP 1077 H=1077:00 +CPU= 57.202
  12:23:11 STEP 1078 H=1078:00 +CPU= 36.107
  12:23:55 STEP 1079 H=1079:00 +CPU= 37.516
  12:25:26 STEP 1080 H=1080:00 +CPU= 68.145
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001080
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001068
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_44.zip
Uploading the intermediate file: upload_file_44.zip
  12:26:14 STEP 1081 H=1081:00 +CPU= 36.601
  12:26:57 STEP 1082 H=1082:00 +CPU= 37.317
  12:28:08 STEP 1083 H=1083:00 +CPU= 57.777
  12:29:05 STEP 1084 H=1084:00 +CPU= 39.000
  12:29:58 STEP 1085 H=1085:00 +CPU= 37.153
  12:31:06 STEP 1086 H=1086:00 +CPU= 58.515
  12:31:51 STEP 1087 H=1087:00 +CPU= 36.783
  12:32:37 STEP 1088 H=1088:00 +CPU= 37.709
  12:33:41 STEP 1089 H=1089:00 +CPU= 57.053
  12:34:26 STEP 1090 H=1090:00 +CPU= 37.465
  12:35:23 STEP 1091 H=1091:00 +CPU= 38.026
  12:36:38 STEP 1092 H=1092:00 +CPU= 63.825
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001092
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001092
  12:37:36 STEP 1093 H=1093:00 +CPU= 38.232
  12:38:24 STEP 1094 H=1094:00 +CPU= 38.341
  12:39:20 STEP 1095 H=1095:00 +CPU= 55.465
  12:40:06 STEP 1096 H=1096:00 +CPU= 37.374
  12:41:06 STEP 1097 H=1097:00 +CPU= 38.324
  12:42:20 STEP 1098 H=1098:00 +CPU= 59.121
  12:43:09 STEP 1099 H=1099:00 +CPU= 36.574
  12:44:01 STEP 1100 H=1100:00 +CPU= 37.569
  12:45:13 STEP 1101 H=1101:00 +CPU= 58.196
  12:46:02 STEP 1102 H=1102:00 +CPU= 37.805
  12:46:44 STEP 1103 H=1103:00 +CPU= 37.031
  12:48:17 STEP 1104 H=1104:00 +CPU= 68.672
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001104
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001092
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_45.zip
Uploading the intermediate file: upload_file_45.zip
  12:49:03 STEP 1105 H=1105:00 +CPU= 36.600
  12:49:50 STEP 1106 H=1106:00 +CPU= 37.736
  12:50:55 STEP 1107 H=1107:00 +CPU= 57.014
  12:51:38 STEP 1108 H=1108:00 +CPU= 37.030
  12:52:20 STEP 1109 H=1109:00 +CPU= 36.220
  12:53:18 STEP 1110 H=1110:00 +CPU= 57.000
  12:53:55 STEP 1111 H=1111:00 +CPU= 35.474
  12:54:49 STEP 1112 H=1112:00 +CPU= 38.796
  12:55:54 STEP 1113 H=1113:00 +CPU= 57.281
  12:56:38 STEP 1114 H=1114:00 +CPU= 37.127
  12:57:20 STEP 1115 H=1115:00 +CPU= 36.274
  12:58:34 STEP 1116 H=1116:00 +CPU= 63.764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001116
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001116
  12:59:26 STEP 1117 H=1117:00 +CPU= 37.703
  13:00:10 STEP 1118 H=1118:00 +CPU= 37.288
  13:01:16 STEP 1119 H=1119:00 +CPU= 57.423
  13:02:06 STEP 1120 H=1120:00 +CPU= 37.525
  13:02:45 STEP 1121 H=1121:00 +CPU= 35.996
  13:03:53 STEP 1122 H=1122:00 +CPU= 58.360
  13:04:40 STEP 1123 H=1123:00 +CPU= 36.930
  13:05:24 STEP 1124 H=1124:00 +CPU= 36.683
  13:06:41 STEP 1125 H=1125:00 +CPU= 58.197
  13:07:21 STEP 1126 H=1126:00 +CPU= 36.853
  13:08:04 STEP 1127 H=1127:00 +CPU= 37.267
  13:09:23 STEP 1128 H=1128:00 +CPU= 66.505
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_46.zip
Uploading the intermediate file: upload_file_46.zip
  13:10:20 STEP 1129 H=1129:00 +CPU= 37.899
  13:11:04 STEP 1130 H=1130:00 +CPU= 37.240
  13:12:09 STEP 1131 H=1131:00 +CPU= 56.639
  13:13:04 STEP 1132 H=1132:00 +CPU= 38.739
  13:13:47 STEP 1133 H=1133:00 +CPU= 36.626
  13:14:52 STEP 1134 H=1134:00 +CPU= 58.118
  13:15:41 STEP 1135 H=1135:00 +CPU= 37.036
  13:16:24 STEP 1136 H=1136:00 +CPU= 37.169
  13:17:49 STEP 1137 H=1137:00 +CPU= 59.679
  13:18:35 STEP 1138 H=1138:00 +CPU= 37.455
  13:19:14 STEP 1139 H=1139:00 +CPU= 35.824
  13:20:30 STEP 1140 H=1140:00 +CPU= 63.982
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001140
  13:21:23 STEP 1141 H=1141:00 +CPU= 37.678
  13:22:02 STEP 1142 H=1142:00 +CPU= 36.623
  13:23:11 STEP 1143 H=1143:00 +CPU= 57.769
  13:24:05 STEP 1144 H=1144:00 +CPU= 38.347
  13:24:51 STEP 1145 H=1145:00 +CPU= 36.741
  13:26:04 STEP 1146 H=1146:00 +CPU= 58.647
  13:26:53 STEP 1147 H=1147:00 +CPU= 36.743
  13:27:38 STEP 1148 H=1148:00 +CPU= 37.344
  13:28:49 STEP 1149 H=1149:00 +CPU= 58.112
  13:29:41 STEP 1150 H=1150:00 +CPU= 38.554
  13:30:21 STEP 1151 H=1151:00 +CPU= 36.993
  13:31:56 STEP 1152 H=1152:00 +CPU= 68.667
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_47.zip
Uploading the intermediate file: upload_file_47.zip
Uploading trickle at timestep: 4147200
  13:32:45 STEP 1153 H=1153:00 +CPU= 37.034
  13:33:30 STEP 1154 H=1154:00 +CPU= 37.023
  13:34:26 STEP 1155 H=1155:00 +CPU= 55.583
  13:35:07 STEP 1156 H=1156:00 +CPU= 36.922
  13:36:07 STEP 1157 H=1157:00 +CPU= 37.266
  13:37:14 STEP 1158 H=1158:00 +CPU= 58.387
  13:37:57 STEP 1159 H=1159:00 +CPU= 36.065
  13:38:40 STEP 1160 H=1160:00 +CPU= 36.647
  13:39:54 STEP 1161 H=1161:00 +CPU= 58.375
  13:40:39 STEP 1162 H=1162:00 +CPU= 37.234
  13:41:23 STEP 1163 H=1163:00 +CPU= 35.956
  13:42:38 STEP 1164 H=1164:00 +CPU= 63.764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001164
  13:43:22 STEP 1165 H=1165:00 +CPU= 36.386
  13:44:05 STEP 1166 H=1166:00 +CPU= 36.765
  13:45:22 STEP 1167 H=1167:00 +CPU= 57.954
  13:46:07 STEP 1168 H=1168:00 +CPU= 37.424
  13:46:51 STEP 1169 H=1169:00 +CPU= 36.270
  13:48:11 STEP 1170 H=1170:00 +CPU= 59.434
  13:48:49 STEP 1171 H=1171:00 +CPU= 35.460
  13:49:33 STEP 1172 H=1172:00 +CPU= 37.372
  13:50:46 STEP 1173 H=1173:00 +CPU= 58.223
  13:51:40 STEP 1174 H=1174:00 +CPU= 38.332
  13:52:22 STEP 1175 H=1175:00 +CPU= 37.329
  13:54:01 STEP 1176 H=1176:00 +CPU= 68.477
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_48.zip
Uploading the intermediate file: upload_file_48.zip
  13:54:48 STEP 1177 H=1177:00 +CPU= 36.822
  13:55:30 STEP 1178 H=1178:00 +CPU= 36.780
  13:56:44 STEP 1179 H=1179:00 +CPU= 58.118
  13:57:27 STEP 1180 H=1180:00 +CPU= 37.319
  13:58:06 STEP 1181 H=1181:00 +CPU= 35.901
  13:59:21 STEP 1182 H=1182:00 +CPU= 58.289
  14:00:06 STEP 1183 H=1183:00 +CPU= 36.545
  14:00:53 STEP 1184 H=1184:00 +CPU= 37.408
  14:01:58 STEP 1185 H=1185:00 +CPU= 57.353
  14:02:45 STEP 1186 H=1186:00 +CPU= 37.777
  14:03:34 STEP 1187 H=1187:00 +CPU= 36.908
  14:04:45 STEP 1188 H=1188:00 +CPU= 63.412
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001188
  14:05:34 STEP 1189 H=1189:00 +CPU= 36.738
  14:06:20 STEP 1190 H=1190:00 +CPU= 38.010
  14:07:24 STEP 1191 H=1191:00 +CPU= 57.056
  14:08:08 STEP 1192 H=1192:00 +CPU= 37.350
  14:09:12 STEP 1193 H=1193:00 +CPU= 37.505
  14:10:24 STEP 1194 H=1194:00 +CPU= 58.489
  14:11:14 STEP 1195 H=1195:00 +CPU= 36.881
  14:11:56 STEP 1196 H=1196:00 +CPU= 36.894
  14:13:05 STEP 1197 H=1197:00 +CPU= 57.878
  14:13:59 STEP 1198 H=1198:00 +CPU= 38.454
  14:14:45 STEP 1199 H=1199:00 +CPU= 37.824
  14:16:15 STEP 1200 H=1200:00 +CPU= 68.205
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_49.zip
Uploading the intermediate file: upload_file_49.zip
  14:17:03 STEP 1201 H=1201:00 +CPU= 36.734
  14:17:44 STEP 1202 H=1202:00 +CPU= 36.738
  14:18:56 STEP 1203 H=1203:00 +CPU= 57.837
  14:19:52 STEP 1204 H=1204:00 +CPU= 39.133
  14:20:33 STEP 1205 H=1205:00 +CPU= 35.908
  14:21:56 STEP 1206 H=1206:00 +CPU= 60.123
  14:22:38 STEP 1207 H=1207:00 +CPU= 36.311
  14:23:17 STEP 1208 H=1208:00 +CPU= 36.710
  14:24:28 STEP 1209 H=1209:00 +CPU= 58.047
  14:25:17 STEP 1210 H=1210:00 +CPU= 37.385
  14:25:56 STEP 1211 H=1211:00 +CPU= 35.532
  14:26:59 STEP 1212 H=1212:00 +CPU= 61.750
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001212
  14:27:40 STEP 1213 H=1213:00 +CPU= 36.142
  14:28:24 STEP 1214 H=1214:00 +CPU= 37.109
  14:29:22 STEP 1215 H=1215:00 +CPU= 55.801
  14:30:04 STEP 1216 H=1216:00 +CPU= 37.375
  14:30:48 STEP 1217 H=1217:00 +CPU= 36.318
  14:31:53 STEP 1218 H=1218:00 +CPU= 57.696
  14:32:45 STEP 1219 H=1219:00 +CPU= 37.327
  14:33:30 STEP 1220 H=1220:00 +CPU= 36.942
  14:34:37 STEP 1221 H=1221:00 +CPU= 57.937
  14:35:22 STEP 1222 H=1222:00 +CPU= 37.077
  14:36:06 STEP 1223 H=1223:00 +CPU= 37.352
  14:37:34 STEP 1224 H=1224:00 +CPU= 67.879
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_50.zip
Uploading the intermediate file: upload_file_50.zip
  14:38:20 STEP 1225 H=1225:00 +CPU= 36.882
  14:39:03 STEP 1226 H=1226:00 +CPU= 36.971
  14:40:16 STEP 1227 H=1227:00 +CPU= 58.042
  14:40:59 STEP 1228 H=1228:00 +CPU= 37.298
  14:41:40 STEP 1229 H=1229:00 +CPU= 35.949
  14:42:41 STEP 1230 H=1230:00 +CPU= 57.364
  14:43:37 STEP 1231 H=1231:00 +CPU= 37.726
  14:44:28 STEP 1232 H=1232:00 +CPU= 37.930
  14:45:35 STEP 1233 H=1233:00 +CPU= 57.806
  14:46:23 STEP 1234 H=1234:00 +CPU= 37.109
  14:47:05 STEP 1235 H=1235:00 +CPU= 36.270
  14:48:19 STEP 1236 H=1236:00 +CPU= 63.590
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001236
  14:49:13 STEP 1237 H=1237:00 +CPU= 36.692
  14:49:57 STEP 1238 H=1238:00 +CPU= 37.562
  14:51:12 STEP 1239 H=1239:00 +CPU= 58.375
  14:51:56 STEP 1240 H=1240:00 +CPU= 37.148
  14:52:37 STEP 1241 H=1241:00 +CPU= 36.103
  14:53:39 STEP 1242 H=1242:00 +CPU= 57.375
  14:54:37 STEP 1243 H=1243:00 +CPU= 38.133
  14:55:26 STEP 1244 H=1244:00 +CPU= 37.560
  14:56:54 STEP 1245 H=1245:00 +CPU= 58.527
  14:57:40 STEP 1246 H=1246:00 +CPU= 37.197
  14:58:22 STEP 1247 H=1247:00 +CPU= 37.086
  14:59:50 STEP 1248 H=1248:00 +CPU= 67.534
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_51.zip
Uploading the intermediate file: upload_file_51.zip
  15:00:47 STEP 1249 H=1249:00 +CPU= 37.910
  15:01:33 STEP 1250 H=1250:00 +CPU= 37.848
  15:02:31 STEP 1251 H=1251:00 +CPU= 55.981
  15:03:13 STEP 1252 H=1252:00 +CPU= 36.820
  15:04:00 STEP 1253 H=1253:00 +CPU= 36.205
  15:04:58 STEP 1254 H=1254:00 +CPU= 56.851
  15:05:35 STEP 1255 H=1255:00 +CPU= 34.823
  15:06:18 STEP 1256 H=1256:00 +CPU= 37.229
  15:07:24 STEP 1257 H=1257:00 +CPU= 57.666
  15:08:15 STEP 1258 H=1258:00 +CPU= 38.221
  15:09:04 STEP 1259 H=1259:00 +CPU= 36.956
  15:10:18 STEP 1260 H=1260:00 +CPU= 63.587
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001260
  15:11:07 STEP 1261 H=1261:00 +CPU= 36.726
  15:11:51 STEP 1262 H=1262:00 +CPU= 37.329
  15:12:59 STEP 1263 H=1263:00 +CPU= 57.830
  15:13:57 STEP 1264 H=1264:00 +CPU= 38.747
  15:14:41 STEP 1265 H=1265:00 +CPU= 36.537
  15:15:55 STEP 1266 H=1266:00 +CPU= 59.050
  15:16:39 STEP 1267 H=1267:00 +CPU= 35.758
  15:17:15 STEP 1268 H=1268:00 +CPU= 35.954
  15:18:39 STEP 1269 H=1269:00 +CPU= 59.900
  15:19:28 STEP 1270 H=1270:00 +CPU= 37.945
  15:20:07 STEP 1271 H=1271:00 +CPU= 36.774
  15:21:37 STEP 1272 H=1272:00 +CPU= 67.833
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_52.zip
Uploading the intermediate file: upload_file_52.zip
  15:22:28 STEP 1273 H=1273:00 +CPU= 36.768
  15:23:14 STEP 1274 H=1274:00 +CPU= 37.043
  15:24:26 STEP 1275 H=1275:00 +CPU= 58.361
  15:25:18 STEP 1276 H=1276:00 +CPU= 38.458
  15:26:06 STEP 1277 H=1277:00 +CPU= 36.788
  15:27:14 STEP 1278 H=1278:00 +CPU= 58.803
  15:27:58 STEP 1279 H=1279:00 +CPU= 36.102
  15:28:44 STEP 1280 H=1280:00 +CPU= 37.904
  15:29:49 STEP 1281 H=1281:00 +CPU= 57.366
  15:30:35 STEP 1282 H=1282:00 +CPU= 37.560
  15:31:16 STEP 1283 H=1283:00 +CPU= 36.144
  15:32:36 STEP 1284 H=1284:00 +CPU= 64.172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001284
  15:33:24 STEP 1285 H=1285:00 +CPU= 36.504
  15:34:07 STEP 1286 H=1286:00 +CPU= 37.126
  15:35:03 STEP 1287 H=1287:00 +CPU= 55.520
  15:35:43 STEP 1288 H=1288:00 +CPU= 36.806
  15:36:30 STEP 1289 H=1289:00 +CPU= 36.798
  15:37:34 STEP 1290 H=1290:00 +CPU= 57.514
  15:38:14 STEP 1291 H=1291:00 +CPU= 35.742
  15:38:55 STEP 1292 H=1292:00 +CPU= 36.528
  15:40:04 STEP 1293 H=1293:00 +CPU= 57.723
  15:40:52 STEP 1294 H=1294:00 +CPU= 37.317
  15:41:33 STEP 1295 H=1295:00 +CPU= 37.244
  15:43:21 STEP 1296 H=1296:00 +CPU= 70.181
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_53.zip
Uploading the intermediate file: upload_file_53.zip
  15:44:05 STEP 1297 H=1297:00 +CPU= 36.368
  15:44:53 STEP 1298 H=1298:00 +CPU= 37.624
  15:46:13 STEP 1299 H=1299:00 +CPU= 58.652
  15:46:56 STEP 1300 H=1300:00 +CPU= 37.113
  15:47:37 STEP 1301 H=1301:00 +CPU= 36.147
  15:48:36 STEP 1302 H=1302:00 +CPU= 57.314
  15:49:17 STEP 1303 H=1303:00 +CPU= 35.594
  15:49:57 STEP 1304 H=1304:00 +CPU= 36.634
  15:51:15 STEP 1305 H=1305:00 +CPU= 58.894
  15:51:59 STEP 1306 H=1306:00 +CPU= 37.022
  15:52:42 STEP 1307 H=1307:00 +CPU= 36.200
  15:54:00 STEP 1308 H=1308:00 +CPU= 64.376
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001308
  15:55:00 STEP 1309 H=1309:00 +CPU= 37.021
  15:55:46 STEP 1310 H=1310:00 +CPU= 37.214
  15:56:51 STEP 1311 H=1311:00 +CPU= 57.369
  15:57:32 STEP 1312 H=1312:00 +CPU= 36.426
  15:58:18 STEP 1313 H=1313:00 +CPU= 36.342
  15:59:27 STEP 1314 H=1314:00 +CPU= 58.249
  16:00:22 STEP 1315 H=1315:00 +CPU= 37.638
  16:01:00 STEP 1316 H=1316:00 +CPU= 36.028
  16:02:15 STEP 1317 H=1317:00 +CPU= 58.557
  16:03:10 STEP 1318 H=1318:00 +CPU= 37.888
  16:03:55 STEP 1319 H=1319:00 +CPU= 36.916
  16:05:13 STEP 1320 H=1320:00 +CPU= 66.457
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_54.zip
Uploading the intermediate file: upload_file_54.zip
  16:05:55 STEP 1321 H=1321:00 +CPU= 36.124
  16:06:43 STEP 1322 H=1322:00 +CPU= 37.037
  16:07:50 STEP 1323 H=1323:00 +CPU= 57.476
  16:08:44 STEP 1324 H=1324:00 +CPU= 38.166
  16:09:41 STEP 1325 H=1325:00 +CPU= 37.845
  16:10:53 STEP 1326 H=1326:00 +CPU= 58.860
  16:11:40 STEP 1327 H=1327:00 +CPU= 36.777
  16:12:34 STEP 1328 H=1328:00 +CPU= 38.795
  16:13:31 STEP 1329 H=1329:00 +CPU= 55.756
  16:14:12 STEP 1330 H=1330:00 +CPU= 36.808
  16:14:56 STEP 1331 H=1331:00 +CPU= 36.722
  16:16:20 STEP 1332 H=1332:00 +CPU= 64.586
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001332
  16:17:02 STEP 1333 H=1333:00 +CPU= 36.142
  16:17:50 STEP 1334 H=1334:00 +CPU= 37.655
  16:19:04 STEP 1335 H=1335:00 +CPU= 57.195
  16:19:45 STEP 1336 H=1336:00 +CPU= 36.649
  16:20:28 STEP 1337 H=1337:00 +CPU= 36.502
  16:21:29 STEP 1338 H=1338:00 +CPU= 57.124
  16:22:24 STEP 1339 H=1339:00 +CPU= 37.399
  16:23:07 STEP 1340 H=1340:00 +CPU= 37.023
  16:24:18 STEP 1341 H=1341:00 +CPU= 58.214
  16:25:11 STEP 1342 H=1342:00 +CPU= 38.292
  16:26:07 STEP 1343 H=1343:00 +CPU= 38.715
  16:27:35 STEP 1344 H=1344:00 +CPU= 67.813
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_55.zip
Uploading the intermediate file: upload_file_55.zip
  16:28:16 STEP 1345 H=1345:00 +CPU= 36.078
  16:29:04 STEP 1346 H=1346:00 +CPU= 37.630
  16:30:02 STEP 1347 H=1347:00 +CPU= 55.772
  16:30:49 STEP 1348 H=1348:00 +CPU= 37.668
  16:31:52 STEP 1349 H=1349:00 +CPU= 37.734
  16:33:03 STEP 1350 H=1350:00 +CPU= 59.052
  16:33:46 STEP 1351 H=1351:00 +CPU= 35.959
  16:34:29 STEP 1352 H=1352:00 +CPU= 37.134
  16:35:37 STEP 1353 H=1353:00 +CPU= 57.568
  16:36:16 STEP 1354 H=1354:00 +CPU= 36.777
  16:36:55 STEP 1355 H=1355:00 +CPU= 36.121
  16:38:08 STEP 1356 H=1356:00 +CPU= 63.878
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001356
  16:38:45 STEP 1357 H=1357:00 +CPU= 36.217
  16:39:26 STEP 1358 H=1358:00 +CPU= 37.390
  16:40:29 STEP 1359 H=1359:00 +CPU= 57.081
  16:41:08 STEP 1360 H=1360:00 +CPU= 36.925
  16:41:47 STEP 1361 H=1361:00 +CPU= 35.975
  16:43:00 STEP 1362 H=1362:00 +CPU= 59.456
  16:43:40 STEP 1363 H=1363:00 +CPU= 38.707
  16:44:21 STEP 1364 H=1364:00 +CPU= 40.737
  16:45:25 STEP 1365 H=1365:00 +CPU= 61.385
  16:46:12 STEP 1366 H=1366:00 +CPU= 36.830
  16:47:06 STEP 1367 H=1367:00 +CPU= 39.267
  16:48:33 STEP 1368 H=1368:00 +CPU= 68.244
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_56.zip
Uploading the intermediate file: upload_file_56.zip
  16:49:15 STEP 1369 H=1369:00 +CPU= 36.609
  16:49:58 STEP 1370 H=1370:00 +CPU= 37.512
  16:51:00 STEP 1371 H=1371:00 +CPU= 56.943
  16:51:40 STEP 1372 H=1372:00 +CPU= 36.964
  16:52:19 STEP 1373 H=1373:00 +CPU= 35.905
  16:53:22 STEP 1374 H=1374:00 +CPU= 62.705
  16:54:02 STEP 1375 H=1375:00 +CPU= 39.657
  16:54:43 STEP 1376 H=1376:00 +CPU= 40.776
  16:55:46 STEP 1377 H=1377:00 +CPU= 62.224
  16:56:27 STEP 1378 H=1378:00 +CPU= 40.577
  16:57:08 STEP 1379 H=1379:00 +CPU= 37.492
  16:58:20 STEP 1380 H=1380:00 +CPU= 64.008
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001380
  16:58:59 STEP 1381 H=1381:00 +CPU= 36.315
  16:59:41 STEP 1382 H=1382:00 +CPU= 37.188
  17:00:43 STEP 1383 H=1383:00 +CPU= 56.837
  17:01:23 STEP 1384 H=1384:00 +CPU= 37.211
  17:02:02 STEP 1385 H=1385:00 +CPU= 36.040
  17:03:00 STEP 1386 H=1386:00 +CPU= 57.460
  17:03:37 STEP 1387 H=1387:00 +CPU= 35.661
  17:04:26 STEP 1388 H=1388:00 +CPU= 41.241
  17:05:29 STEP 1389 H=1389:00 +CPU= 62.391
  17:06:10 STEP 1390 H=1390:00 +CPU= 40.500
  17:06:51 STEP 1391 H=1391:00 +CPU= 41.008
  17:08:16 STEP 1392 H=1392:00 +CPU= 72.326
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_57.zip
Uploading the intermediate file: upload_file_57.zip
Uploading trickle at timestep: 5011200
  17:08:56 STEP 1393 H=1393:00 +CPU= 36.173
  17:09:37 STEP 1394 H=1394:00 +CPU= 37.308
  17:10:50 STEP 1395 H=1395:00 +CPU= 58.483
  17:11:31 STEP 1396 H=1396:00 +CPU= 37.261
  17:12:12 STEP 1397 H=1397:00 +CPU= 36.463
  17:13:25 STEP 1398 H=1398:00 +CPU= 59.603
  17:14:05 STEP 1399 H=1399:00 +CPU= 36.187
  17:14:44 STEP 1400 H=1400:00 +CPU= 36.853
  17:15:45 STEP 1401 H=1401:00 +CPU= 60.488
  17:16:26 STEP 1402 H=1402:00 +CPU= 40.485
  17:17:05 STEP 1403 H=1403:00 +CPU= 39.302
  17:18:22 STEP 1404 H=1404:00 +CPU= 64.291
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001404
  17:19:01 STEP 1405 H=1405:00 +CPU= 36.251
  17:19:42 STEP 1406 H=1406:00 +CPU= 37.411
  17:20:46 STEP 1407 H=1407:00 +CPU= 57.199
  17:21:33 STEP 1408 H=1408:00 +CPU= 37.559
  17:22:11 STEP 1409 H=1409:00 +CPU= 36.156
  17:23:09 STEP 1410 H=1410:00 +CPU= 57.113
  17:23:52 STEP 1411 H=1411:00 +CPU= 36.386
  17:24:38 STEP 1412 H=1412:00 +CPU= 40.032
  17:25:42 STEP 1413 H=1413:00 +CPU= 62.660
  17:26:22 STEP 1414 H=1414:00 +CPU= 40.649
  17:27:04 STEP 1415 H=1415:00 +CPU= 41.206
  17:28:23 STEP 1416 H=1416:00 +CPU= 73.156
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_58.zip
Uploading the intermediate file: upload_file_58.zip
  17:29:05 STEP 1417 H=1417:00 +CPU= 36.297
  17:29:51 STEP 1418 H=1418:00 +CPU= 37.742
  17:31:00 STEP 1419 H=1419:00 +CPU= 58.223
  17:31:39 STEP 1420 H=1420:00 +CPU= 37.096
  17:32:24 STEP 1421 H=1421:00 +CPU= 36.968
  17:33:29 STEP 1422 H=1422:00 +CPU= 58.745
  17:34:08 STEP 1423 H=1423:00 +CPU= 36.198
  17:34:48 STEP 1424 H=1424:00 +CPU= 37.009
  17:35:56 STEP 1425 H=1425:00 +CPU= 60.268
  17:36:37 STEP 1426 H=1426:00 +CPU= 40.679
  17:37:17 STEP 1427 H=1427:00 +CPU= 39.550
  17:38:29 STEP 1428 H=1428:00 +CPU= 70.115
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001428
  17:39:10 STEP 1429 H=1429:00 +CPU= 39.769
  17:39:51 STEP 1430 H=1430:00 +CPU= 39.177
  17:40:56 STEP 1431 H=1431:00 +CPU= 56.910
  17:41:36 STEP 1432 H=1432:00 +CPU= 37.383
  17:42:17 STEP 1433 H=1433:00 +CPU= 36.718
  17:43:23 STEP 1434 H=1434:00 +CPU= 58.464
  17:44:06 STEP 1435 H=1435:00 +CPU= 36.475
  17:44:48 STEP 1436 H=1436:00 +CPU= 37.593
  17:45:51 STEP 1437 H=1437:00 +CPU= 55.297
  17:46:31 STEP 1438 H=1438:00 +CPU= 36.617
  17:47:19 STEP 1439 H=1439:00 +CPU= 43.752
  17:48:46 STEP 1440 H=1440:00 +CPU= 79.198
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961_59.zip
Uploading the intermediate file: upload_file_59.zip
  17:49:29 STEP 1441 H=1441:00 +CPU= 43.151
  17:50:14 STEP 1442 H=1442:00 +CPU= 44.023
  17:51:23 STEP 1443 H=1443:00 +CPU= 66.174
  17:52:06 STEP 1444 H=1444:00 +CPU= 42.600
  17:52:54 STEP 1445 H=1445:00 +CPU= 40.911
  17:54:01 STEP 1446 H=1446:00 +CPU= 61.844
  17:54:47 STEP 1447 H=1447:00 +CPU= 39.531
  17:55:33 STEP 1448 H=1448:00 +CPU= 40.475
  17:56:43 STEP 1449 H=1449:00 +CPU= 62.935
  17:57:30 STEP 1450 H=1450:00 +CPU= 41.331
  17:58:15 STEP 1451 H=1451:00 +CPU= 39.799
  17:59:31 STEP 1452 H=1452:00 +CPU= 67.809
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001452
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1995050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0091
(argv4) batchid: 964
(argv5) wuid: 12180735
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12180735
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_35f95cb6625606525bcaf1461b63cfe4 to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0091_1995050100_123_964_12180735.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0091_1995050100_123_964_12180735.zip
ic_ancil_file: ic_ancil_12180735
ifsdata_file: ifsdata_12180735
climate_data_file: clim_data_12180735
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_f9cda84115fe368db6c2504422b9007e to: /var/lib/boinc-client/slots/2/ic_ancil_12180735.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12180735.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_737205b31ba17382165266acd48b2360 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12180735.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12180735.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12180735.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12180735.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12180735.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12180735.xml
-- Model is restarting --
Adjusting last_iter, 1453, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12180735.xml
last_cpu_time: 69817
upload_file_number: 60
last_iter: 1439
last_upload: 5184000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0091_1995050100_123_964_12180735_0_r468669961
The child process has been launched with process id: 744340
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:mediahive:1:1:744340:744340] [20221225:080005:1671951605:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  08:01:28 STEP 1440 H=1440:00 +CPU= 73.500
  08:02:41 STEP 1441 H=1441:00 +CPU= 45.004
  08:03:40 STEP 1442 H=1442:00 +CPU= 43.521
  08:05:08 STEP 1443 H=1443:00 +CPU= 65.084
  08:06:07 STEP 1444 H=1444:00 +CPU= 42.533
  08:07:04 STEP 1445 H=1445:00 +CPU= 40.613
  08:08:10 STEP 1446 H=1446:00 +CPU= 65.587
  08:08:52 STEP 1447 H=1447:00 +CPU= 41.717
  08:09:37 STEP 1448 H=1448:00 +CPU= 44.347
  08:10:44 STEP 1449 H=1449:00 +CPU= 66.991
  08:11:27 STEP 1450 H=1450:00 +CPU= 43.030
  08:12:08 STEP 1451 H=1451:00 +CPU= 40.501
  08:13:18 STEP 1452 H=1452:00 +CPU= 64.243
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001452
  08:13:57 STEP 1453 H=1453:00 +CPU= 36.551
  08:14:36 STEP 1454 H=1454:00 +CPU= 37.108
  08:15:41 STEP 1455 H=1455:00 +CPU= 59.199
  08:16:29 STEP 1456 H=1456:00 +CPU= 39.540
  08:17:10 STEP 1457 H=1457:00 +CPU= 37.746
  08:18:19 STEP 1458 H=1458:00 +CPU= 62.472
  08:19:03 STEP 1459 H=1459:00 +CPU= 39.598
  08:19:50 STEP 1460 H=1460:00 +CPU= 42.927
  08:20:57 STEP 1461 H=1461:00 +CPU= 66.654
  08:21:41 STEP 1462 H=1462:00 +CPU= 43.810
  08:22:26 STEP 1463 H=1463:00 +CPU= 43.004
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:52:42 1537254 22265721 oifs_43r3_ps_0091_1995050100_123_964_12180735_0 5,011,200 66,995 0.0134
09 May 2023 15:52:21 1537254 22265721 oifs_43r3_ps_0091_1995050100_123_964_12180735_0 4,147,200 56,248 0.0136
09 May 2023 15:48:04 1537254 22265721 oifs_43r3_ps_0091_1995050100_123_964_12180735_0 864,000 10,609 0.0123


©2024 cpdn.org