climateprediction.net home page
Task 22290457

Task 22290457

Name oifs_43r3_ps_0332_2018050100_123_987_12203976_0
Workunit 12203976
Created 24 Dec 2022, 20:22:23 UTC
Sent 17 Jan 2023, 16:11:13 UTC
Report deadline 18 Mar 2023, 16:11:13 UTC
Received 18 Jan 2023, 10:36:25 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1534746
Run time 4 hours 3 min 12 sec
CPU time 1 hours 46 min 21 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 5.61 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,737.05 MB
Peak swap size 4,978.07 MB
Peak disk usage 2,017.93 MB
Stderr 
<core_client_version>7.20.5</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:16:21 STEP   72 H=  72:00 +CPU= 65.573
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:18:16 STEP   73 H=  73:00 +CPU= 25.999
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:18:44 STEP   74 H=  74:00 +CPU= 16.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:19:13 STEP   75 H=  75:00 +CPU= 26.439
  04:19:28 STEP   76 H=  76:00 +CPU= 15.633
  04:19:47 STEP   77 H=  77:00 +CPU= 18.646
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  04:20:20 STEP   78 H=  78:00 +CPU= 29.986
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:48 STEP   79 H=  79:00 +CPU=120.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:42:16 STEP   80 H=  80:00 +CPU= 17.358
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:42:42 STEP   81 H=  81:00 +CPU= 24.238
  05:42:58 STEP   82 H=  82:00 +CPU= 15.615
  05:43:12 STEP   83 H=  83:00 +CPU= 14.230
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:43:40 STEP   84 H=  84:00 +CPU= 26.306
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000084
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000084
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000084
  05:43:54 STEP   85 H=  85:00 +CPU= 13.628
  05:44:08 STEP   86 H=  86:00 +CPU= 13.972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:34 STEP   87 H=  87:00 +CPU= 24.244
  05:44:49 STEP   88 H=  88:00 +CPU= 14.742
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:45:07 STEP   89 H=  89:00 +CPU= 15.616
  05:45:34 STEP   90 H=  90:00 +CPU= 27.336
  05:45:49 STEP   91 H=  91:00 +CPU= 14.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:05 STEP   92 H=  92:00 +CPU= 14.421
  05:46:32 STEP   93 H=  93:00 +CPU= 26.666
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:51 STEP   94 H=  94:00 +CPU= 17.125
  05:47:05 STEP   95 H=  95:00 +CPU= 13.976
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:35 STEP   96 H=  96:00 +CPU= 27.688
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000096
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000096
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000084
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000084
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000084
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_3.zip
Uploading the intermediate file: upload_file_3.zip
  05:47:50 STEP   97 H=  97:00 +CPU= 15.663
  05:48:04 STEP   98 H=  98:00 +CPU= 13.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:33 STEP   99 H=  99:00 +CPU= 26.784
  05:48:49 STEP  100 H= 100:00 +CPU= 16.073
  05:49:04 STEP  101 H= 101:00 +CPU= 15.051
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:49:30 STEP  102 H= 102:00 +CPU= 23.513
  05:49:45 STEP  103 H= 103:00 +CPU= 15.412
  05:50:01 STEP  104 H= 104:00 +CPU= 15.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:26 STEP  105 H= 105:00 +CPU= 23.231
  05:50:41 STEP  106 H= 106:00 +CPU= 15.298
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:59 STEP  107 H= 107:00 +CPU= 15.531
  05:51:24 STEP  108 H= 108:00 +CPU= 25.035
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000108
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000108
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000108
  05:51:37 STEP  109 H= 109:00 +CPU= 13.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:51:55 STEP  110 H= 110:00 +CPU= 15.358
  05:52:19 STEP  111 H= 111:00 +CPU= 24.260
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:52:36 STEP  112 H= 112:00 +CPU= 14.833
  05:52:51 STEP  113 H= 113:00 +CPU= 15.483
  05:53:16 STEP  114 H= 114:00 +CPU= 25.077
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:33 STEP  115 H= 115:00 +CPU= 14.431
  05:53:48 STEP  116 H= 116:00 +CPU= 15.053
  05:54:12 STEP  117 H= 117:00 +CPU= 24.183
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:54:29 STEP  118 H= 118:00 +CPU= 14.730
  05:54:45 STEP  119 H= 119:00 +CPU= 16.178
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:17 STEP  120 H= 120:00 +CPU= 29.931
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000120
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000120
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000108
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_4.zip
Uploading the intermediate file: upload_file_4.zip
  05:55:32 STEP  121 H= 121:00 +CPU= 14.672
  05:55:49 STEP  122 H= 122:00 +CPU= 17.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:16 STEP  123 H= 123:00 +CPU= 24.633
  05:56:32 STEP  124 H= 124:00 +CPU= 15.773
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:49 STEP  125 H= 125:00 +CPU= 15.741
  05:57:12 STEP  126 H= 126:00 +CPU= 23.073
  05:57:28 STEP  127 H= 127:00 +CPU= 15.436
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:57:46 STEP  128 H= 128:00 +CPU= 15.796
  05:58:09 STEP  129 H= 129:00 +CPU= 23.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:28 STEP  130 H= 130:00 +CPU= 17.292
  05:58:44 STEP  131 H= 131:00 +CPU= 15.333
  05:59:09 STEP  132 H= 132:00 +CPU= 25.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000132
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000132
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000132
  05:59:28 STEP  133 H= 133:00 +CPU= 16.587
  05:59:43 STEP  134 H= 134:00 +CPU= 15.190
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:00:08 STEP  135 H= 135:00 +CPU= 22.886
  06:00:22 STEP  136 H= 136:00 +CPU= 13.969
  06:00:38 STEP  137 H= 137:00 +CPU= 16.066
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:01:06 STEP  138 H= 138:00 +CPU= 26.108
  06:01:22 STEP  139 H= 139:00 +CPU= 15.891
  06:01:36 STEP  140 H= 140:00 +CPU= 14.139
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:02:02 STEP  141 H= 141:00 +CPU= 23.262
  06:02:19 STEP  142 H= 142:00 +CPU= 17.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:02:38 STEP  143 H= 143:00 +CPU= 16.563
  06:03:08 STEP  144 H= 144:00 +CPU= 29.921
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000144
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000144
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000132
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000132
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000132
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_5.zip
Uploading the intermediate file: upload_file_5.zip
  06:03:21 STEP  145 H= 145:00 +CPU= 13.839
  06:03:35 STEP  146 H= 146:00 +CPU= 14.062
  06:04:01 STEP  147 H= 147:00 +CPU= 25.615
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:19 STEP  148 H= 148:00 +CPU= 16.343
  06:04:34 STEP  149 H= 149:00 +CPU= 15.028
  06:04:57 STEP  150 H= 150:00 +CPU= 22.540
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:05:14 STEP  151 H= 151:00 +CPU= 14.610
  06:05:28 STEP  152 H= 152:00 +CPU= 14.529
  06:05:52 STEP  153 H= 153:00 +CPU= 24.069
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:06:10 STEP  154 H= 154:00 +CPU= 15.848
  06:06:25 STEP  155 H= 155:00 +CPU= 15.314
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:06:54 STEP  156 H= 156:00 +CPU= 27.013
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000156
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000156
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000156
  06:07:08 STEP  157 H= 157:00 +CPU= 13.199
  06:07:22 STEP  158 H= 158:00 +CPU= 14.314
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:07:48 STEP  159 H= 159:00 +CPU= 24.008
  06:08:05 STEP  160 H= 160:00 +CPU= 17.347
  06:08:20 STEP  161 H= 161:00 +CPU= 14.492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:08:50 STEP  162 H= 162:00 +CPU= 28.473
  06:09:04 STEP  163 H= 163:00 +CPU= 13.963
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:09:22 STEP  164 H= 164:00 +CPU= 15.861
  06:09:47 STEP  165 H= 165:00 +CPU= 25.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:10:06 STEP  166 H= 166:00 +CPU= 16.997
  06:10:22 STEP  167 H= 167:00 +CPU= 15.849
  06:10:50 STEP  168 H= 168:00 +CPU= 27.950
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000168
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000168
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000156
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000156
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000156
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_6.zip
Uploading the intermediate file: upload_file_6.zip
  06:11:08 STEP  169 H= 169:00 +CPU= 15.878
  06:11:24 STEP  170 H= 170:00 +CPU= 15.902
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:11:51 STEP  171 H= 171:00 +CPU= 24.387
  06:12:06 STEP  172 H= 172:00 +CPU= 15.261
  06:12:21 STEP  173 H= 173:00 +CPU= 15.094
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:12:50 STEP  174 H= 174:00 +CPU= 26.915
  06:13:05 STEP  175 H= 175:00 +CPU= 14.865
  06:13:19 STEP  176 H= 176:00 +CPU= 13.964
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:13:47 STEP  177 H= 177:00 +CPU= 25.919
  06:14:03 STEP  178 H= 178:00 +CPU= 16.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:14:20 STEP  179 H= 179:00 +CPU= 15.278
  06:14:44 STEP  180 H= 180:00 +CPU= 24.060
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000180
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000180
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000180
  06:14:58 STEP  181 H= 181:00 +CPU= 14.194
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:15:16 STEP  182 H= 182:00 +CPU= 15.673
  06:15:42 STEP  183 H= 183:00 +CPU= 25.743
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:15:59 STEP  184 H= 184:00 +CPU= 15.288
  06:16:15 STEP  185 H= 185:00 +CPU= 16.055
  06:16:40 STEP  186 H= 186:00 +CPU= 24.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:16:57 STEP  187 H= 187:00 +CPU= 15.770
  06:17:13 STEP  188 H= 188:00 +CPU= 15.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:17:42 STEP  189 H= 189:00 +CPU= 26.705
  06:17:57 STEP  190 H= 190:00 +CPU= 14.791
  06:18:11 STEP  191 H= 191:00 +CPU= 13.859
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:18:40 STEP  192 H= 192:00 +CPU= 27.698
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000192
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000192
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000180
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000180
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000180
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_7.zip
Uploading the intermediate file: upload_file_7.zip
  06:18:55 STEP  193 H= 193:00 +CPU= 14.108
  06:19:09 STEP  194 H= 194:00 +CPU= 14.520
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:37 STEP  195 H= 195:00 +CPU= 25.553
  06:19:52 STEP  196 H= 196:00 +CPU= 15.456
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:20:09 STEP  197 H= 197:00 +CPU= 15.091
  06:20:35 STEP  198 H= 198:00 +CPU= 25.671
  06:20:50 STEP  199 H= 199:00 +CPU= 14.682
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:21:09 STEP  200 H= 200:00 +CPU= 17.236
  06:21:33 STEP  201 H= 201:00 +CPU= 23.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:21:50 STEP  202 H= 202:00 +CPU= 15.305
  06:22:04 STEP  203 H= 203:00 +CPU= 14.256
  06:22:31 STEP  204 H= 204:00 +CPU= 26.729
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000204
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000204
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000204
  06:22:51 STEP  205 H= 205:00 +CPU= 17.910
  06:23:06 STEP  206 H= 206:00 +CPU= 15.209
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:23:32 STEP  207 H= 207:00 +CPU= 24.001
  06:23:47 STEP  208 H= 208:00 +CPU= 15.062
  06:24:02 STEP  209 H= 209:00 +CPU= 15.054
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:24:30 STEP  210 H= 210:00 +CPU= 25.377
  06:24:44 STEP  211 H= 211:00 +CPU= 13.843
  06:24:58 STEP  212 H= 212:00 +CPU= 14.698
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:25:24 STEP  213 H= 213:00 +CPU= 23.931
  06:25:39 STEP  214 H= 214:00 +CPU= 14.421
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:25:58 STEP  215 H= 215:00 +CPU= 17.498
  06:26:28 STEP  216 H= 216:00 +CPU= 29.646
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000216
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000216
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000204
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000204
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000204
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_8.zip
Uploading the intermediate file: upload_file_8.zip
  06:26:43 STEP  217 H= 217:00 +CPU= 15.200
  06:26:59 STEP  218 H= 218:00 +CPU= 16.143
  06:27:24 STEP  219 H= 219:00 +CPU= 25.102
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:27:44 STEP  220 H= 220:00 +CPU= 17.894
  06:28:01 STEP  221 H= 221:00 +CPU= 16.838
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:28:31 STEP  222 H= 222:00 +CPU= 28.237
  06:28:46 STEP  223 H= 223:00 +CPU= 14.481
  06:29:00 STEP  224 H= 224:00 +CPU= 14.391
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:29:26 STEP  225 H= 225:00 +CPU= 23.396
  06:29:41 STEP  226 H= 226:00 +CPU= 15.305
  06:29:55 STEP  227 H= 227:00 +CPU= 13.941
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:30:23 STEP  228 H= 228:00 +CPU= 26.594
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000228
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000228
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000228
  06:30:37 STEP  229 H= 229:00 +CPU= 13.761
  06:30:52 STEP  230 H= 230:00 +CPU= 14.590
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:31:19 STEP  231 H= 231:00 +CPU= 24.792
  06:31:35 STEP  232 H= 232:00 +CPU= 16.383
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:31:54 STEP  233 H= 233:00 +CPU= 17.219
  06:32:17 STEP  234 H= 234:00 +CPU= 23.200
  06:32:32 STEP  235 H= 235:00 +CPU= 14.580
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:32:51 STEP  236 H= 236:00 +CPU= 16.731
  06:33:19 STEP  237 H= 237:00 +CPU= 27.875
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:33:39 STEP  238 H= 238:00 +CPU= 18.822
  06:33:55 STEP  239 H= 239:00 +CPU= 16.031
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:34:30 STEP  240 H= 240:00 +CPU= 32.331
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000240
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000240
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000228
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000228
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000228
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_9.zip
Uploading the intermediate file: upload_file_9.zip
  06:34:46 STEP  241 H= 241:00 +CPU= 16.364
  06:35:02 STEP  242 H= 242:00 +CPU= 15.714
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:35:28 STEP  243 H= 243:00 +CPU= 24.149
  06:35:43 STEP  244 H= 244:00 +CPU= 14.941
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:36:01 STEP  245 H= 245:00 +CPU= 16.011
  06:36:26 STEP  246 H= 246:00 +CPU= 24.944
  06:36:41 STEP  247 H= 247:00 +CPU= 14.992
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:36:58 STEP  248 H= 248:00 +CPU= 15.413
  06:37:24 STEP  249 H= 249:00 +CPU= 25.290
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:37:45 STEP  250 H= 250:00 +CPU= 18.998
  06:38:00 STEP  251 H= 251:00 +CPU= 15.493
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:38:28 STEP  252 H= 252:00 +CPU= 25.624
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000252
  06:38:43 STEP  253 H= 253:00 +CPU= 14.968
  06:38:58 STEP  254 H= 254:00 +CPU= 15.692
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:39:25 STEP  255 H= 255:00 +CPU= 24.575
  06:39:40 STEP  256 H= 256:00 +CPU= 15.312
  06:39:55 STEP  257 H= 257:00 +CPU= 14.806
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:40:23 STEP  258 H= 258:00 +CPU= 25.716
  06:40:39 STEP  259 H= 259:00 +CPU= 16.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:40:59 STEP  260 H= 260:00 +CPU= 17.376
  06:41:22 STEP  261 H= 261:00 +CPU= 23.509
  06:41:36 STEP  262 H= 262:00 +CPU= 14.073
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:41:53 STEP  263 H= 263:00 +CPU= 14.651
  06:42:23 STEP  264 H= 264:00 +CPU= 29.835
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000264
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000264
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMGGhq0f+000252
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMSHhq0f+000252
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976/ICMUAhq0f+000252
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539_10.zip
Uploading the intermediate file: upload_file_10.zip
  06:42:38 STEP  265 H= 265:00 +CPU= 15.865
  06:42:54 STEP  266 H= 266:00 +CPU= 15.882
  06:43:21 STEP  267 H= 267:00 +CPU= 26.303
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:43:39 STEP  268 H= 268:00 +CPU= 15.897
  06:43:54 STEP  269 H= 269:00 +CPU= 15.687
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:44:22 STEP  270 H= 270:00 +CPU= 25.289
  06:44:35 STEP  271 H= 271:00 +CPU= 13.816
  06:44:50 STEP  272 H= 272:00 +CPU= 14.964
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:45:16 STEP  273 H= 273:00 +CPU= 23.639
  06:45:31 STEP  274 H= 274:00 +CPU= 14.727
  06:45:44 STEP  275 H= 275:00 +CPU= 13.002
  06:46:12 STEP  276 H= 276:00 +CPU= 27.462
Moving to projects directory: /var/lib/boinc/slots/7/ICMGGhq0f+000276
Moving to projects directory: /var/lib/boinc/slots/7/ICMSHhq0f+000276
Moving to projects directory: /var/lib/boinc/slots/7/ICMUAhq0f+000276
  06:46:26 STEP  277 H= 277:00 +CPU= 14.033
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2018050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0332
(argv4) batchid: 987
(argv5) wuid: 12203976
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/7
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
  07:10:49 STEP  278 H= 278:00 +CPU= 25.723
Copying the namelist files from: ../../projects/climateprediction.net/jf_1c0bc4761e7510ce5fec64f0eb057809 to: /var/lib/boinc/slots/7/oifs_43r3_ps_0332_2018050100_123_987_12203976.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/7/oifs_43r3_ps_0332_2018050100_123_987_12203976.zip
ic_ancil_file: ic_ancil_12203976
ifsdata_file: ifsdata_12203976
climate_data_file: clim_data_12203976
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_f59754219e7224d5a449e3345b85c3cd to: /var/lib/boinc/slots/7/ic_ancil_12203976.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/7/ic_ancil_12203976.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6994ab4532f7935e5109a0b2c4796caf to: /var/lib/boinc/slots/7/ifsdata/ifsdata_12203976.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/7/ifsdata/ifsdata_12203976.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/7/159l_2/clim_data_12203976.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/7/159l_2/clim_data_12203976.zip
Checking for progress XML file: /var/lib/boinc/slots/7/progress_file_12203976.xml
Opened progress file ok : /var/lib/boinc/slots/7/progress_file_12203976.xml
-- Model is restarting --
Adjusting last_iter, 278, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/7/progress_file_12203976.xml
last_cpu_time: 5726
upload_file_number: 11
last_iter: 263
last_upload: 950400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539
The child process has been launched with process id: 9525
Executing the command: /var/lib/boinc/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9525:9525] [20230118:071053:1674000653:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2018050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0332
(argv4) batchid: 987
(argv5) wuid: 12203976
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/7
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12203976
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/7/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1c0bc4761e7510ce5fec64f0eb057809 to: /var/lib/boinc/slots/7/oifs_43r3_ps_0332_2018050100_123_987_12203976.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/7/oifs_43r3_ps_0332_2018050100_123_987_12203976.zip
ic_ancil_file: ic_ancil_12203976
ifsdata_file: ifsdata_12203976
climate_data_file: clim_data_12203976
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_f59754219e7224d5a449e3345b85c3cd to: /var/lib/boinc/slots/7/ic_ancil_12203976.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/7/ic_ancil_12203976.zip
  07:33:09 STEP  264 H= 264:00 +CPU=101.444
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_6994ab4532f7935e5109a0b2c4796caf to: /var/lib/boinc/slots/7/ifsdata/ifsdata_12203976.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/7/ifsdata/ifsdata_12203976.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/7/159l_2/clim_data_12203976.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/7/159l_2/clim_data_12203976.zip
Checking for progress XML file: /var/lib/boinc/slots/7/progress_file_12203976.xml
Opened progress file ok : /var/lib/boinc/slots/7/progress_file_12203976.xml
-- Model is restarting --
Adjusting last_iter, 264, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/7/progress_file_12203976.xml
last_cpu_time: 5792
upload_file_number: 11
last_iter: 263
last_upload: 950400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0332_2018050100_123_987_12203976_0_r444741539
The child process has been launched with process id: 9669
Executing the command: /var/lib/boinc/slots/7/oifs_43r3_model.exe
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:073734:1674002254:29.169] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:073741:1674002261:74.097] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074531:1674002731:500.645] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074538:1674002738:508.765] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074543:1674002743:528.460] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074602:1674002762:532.515] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074612:1674002772:542.935] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074618:1674002778:628.656] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074757:1674002877:653.371] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074833:1674002913:681.834] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074836:1674002916:695.568] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074925:1674002965:736.698] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074937:1674002977:747.419] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074944:1674002984:754.230] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074947:1674002987:757.666] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:074955:1674002995:766.179] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075108:1674003068:839.296] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075121:1674003081:850.581] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075137:1674003097:872.959] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075204:1674003124:893.669] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075241:1674003161:946.331] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075340:1674003220:993.616] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075409:1674003249:1019.420] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075418:1674003258:1031.445] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075434:1674003274:1044.181] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075441:1674003281:1077.146] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075528:1674003328:1115.735] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075552:1674003352:1128.377] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075606:1674003366:1135.067] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:vm9:1:1:9669:9669] [20230118:075648:1674003408:1178.409] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:43:14 STEP  264 H= 264:00 +CPU=184.566
  12:43:34 STEP  265 H= 265:00 +CPU= 19.676
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net