Name | oifs_43r3_bl_a0zu_2016092300_15_991_12208077_0 |
Workunit | 12208077 |
Created | 13 Feb 2023, 14:23:28 UTC |
Sent | 13 Feb 2023, 14:27:43 UTC |
Report deadline | 14 Apr 2023, 14:27:43 UTC |
Received | 14 Feb 2023, 1:22:28 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1538161 |
Run time | 2 hours 29 min 26 sec |
CPU time | 2 hours 1 min 56 sec |
Validate state | Invalid |
Credit | 0.00 |
Device peak FLOPS | 7.28 GFLOPS |
Application version | OpenIFS 43r3 Baroclinic Lifecycle v1.11 x86_64-pc-linux-gnu |
Peak working set size | 5,058.36 MB |
Peak swap size | 5,930.79 MB |
Peak disk usage | 1,291.42 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> ocess Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:16:18 STEP 197 H= 49:15 +CPU= 20.419 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:16:48 STEP 198 H= 49:30 +CPU= 24.364 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:17:20 STEP 199 H= 49:45 +CPU= 24.886 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:17:50 STEP 200 H= 50:00 +CPU= 24.232 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:18:21 STEP 201 H= 50:15 +CPU= 24.678 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:18:51 STEP 202 H= 50:30 +CPU= 24.257 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:19:22 STEP 203 H= 50:45 +CPU= 24.225 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:20:01 STEP 204 H= 51:00 +CPU= 30.323 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000204 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000204 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000204 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:20:31 STEP 205 H= 51:15 +CPU= 24.615 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:21:02 STEP 206 H= 51:30 +CPU= 24.032 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:21:32 STEP 207 H= 51:45 +CPU= 24.398 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:22:03 STEP 208 H= 52:00 +CPU= 24.474 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:22:32 STEP 209 H= 52:15 +CPU= 23.075 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:23:01 STEP 210 H= 52:30 +CPU= 22.270 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:23:30 STEP 211 H= 52:45 +CPU= 23.253 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:24:03 STEP 212 H= 53:00 +CPU= 26.593 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:24:37 STEP 213 H= 53:15 +CPU= 25.533 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:25:07 STEP 214 H= 53:30 +CPU= 24.020 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:25:37 STEP 215 H= 53:45 +CPU= 23.455 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:26:13 STEP 216 H= 54:00 +CPU= 29.206 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000216 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000216 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000216 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:26:44 STEP 217 H= 54:15 +CPU= 24.623 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:27:14 STEP 218 H= 54:30 +CPU= 24.073 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:27:44 STEP 219 H= 54:45 +CPU= 23.423 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:28:12 STEP 220 H= 55:00 +CPU= 22.542 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:28:41 STEP 221 H= 55:15 +CPU= 22.476 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:29:09 STEP 222 H= 55:30 +CPU= 21.727 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:29:34 STEP 223 H= 55:45 +CPU= 21.035 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:30:04 STEP 224 H= 56:00 +CPU= 23.134 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:30:32 STEP 225 H= 56:15 +CPU= 21.366 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:30:58 STEP 226 H= 56:30 +CPU= 22.689 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:31:28 STEP 227 H= 56:45 +CPU= 23.094 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:32:03 STEP 228 H= 57:00 +CPU= 28.856 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000228 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000228 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000228 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:32:32 STEP 229 H= 57:15 +CPU= 22.462 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:33:01 STEP 230 H= 57:30 +CPU= 23.134 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:33:32 STEP 231 H= 57:45 +CPU= 24.449 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:34:02 STEP 232 H= 58:00 +CPU= 24.052 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:34:33 STEP 233 H= 58:15 +CPU= 23.893 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:35:02 STEP 234 H= 58:30 +CPU= 23.220 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:35:32 STEP 235 H= 58:45 +CPU= 23.755 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:36:02 STEP 236 H= 59:00 +CPU= 22.994 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:36:28 STEP 237 H= 59:15 +CPU= 21.637 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:36:54 STEP 238 H= 59:30 +CPU= 21.955 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:37:22 STEP 239 H= 59:45 +CPU= 21.826 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:38:02 STEP 240 H= 60:00 +CPU= 31.448 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000240 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000240 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000240 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:38:29 STEP 241 H= 60:15 +CPU= 21.911 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:38:59 STEP 242 H= 60:30 +CPU= 23.995 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:39:28 STEP 243 H= 60:45 +CPU= 23.456 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:39:58 STEP 244 H= 61:00 +CPU= 23.605 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:40:29 STEP 245 H= 61:15 +CPU= 24.753 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:40:57 STEP 246 H= 61:30 +CPU= 21.659 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:41:24 STEP 247 H= 61:45 +CPU= 22.846 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:41:51 STEP 248 H= 62:00 +CPU= 23.263 Quit request received from BOINC client, ending the child process >>> Printing last 70 lines from file: NODE.001_01 I I Sens. heat rain II 0.00000 I I I I I I I Sens. heat snow II 0.00000 I I I I I DDH--------------------------------------------------------------------------------------------------I I PREC- Pot. and kin. E.II 0.00000 I 0.00000 I I I I DDH--------------------------------------------------------------------------------------------------I I Orographic drag II 0.00000 I 0.00000 I I I I DDH--------------------------------------------------------------------------------------------------I I I Balance II 0.00000 I I I I I I ADIAB- of which II I I I I I I I Hor. Flux of PEII -0.00000 I I I I I I I Vert.Flux of PEII 0.00000 I I I I I I I Work Pres.G.F II I -0.00824 I I I I I I Conv. Ad.-Int. II 0.00824 I I I I I DDH--------------------------------------------------------------------------------------------------I I DYN- Horizont. Flux II 0.00043 I -0.00031 I 0.00765 I 0.00000 I -0.00000 I I I Vertical Flux II 0.00000 I 0.00000 I 0.00000 I 0.00000 I 0.00000 I DDH--------------------------------------------------------------------------------------------------I DDH---------------------------------------------------------------------------------------------------------------------------------- I - Mass balance (10**(-6) KG/M**2) I INITIAL I HORIZ. FLUX I DYN. FLUX I DYN. FLUX I PHYS. FLUX I PHYS. FLUX I CURRENT I I I RESIDUAL I I MASS I I TOA I SURFACE I TOA I SURFACE I MASS I I I I ---------------------------------------------------------------------------------------------------------- I ---------------- - - - - - - - - I - - - 10206523039. I 0. . 0. . 0. . 0. . 0. I 10206522993. - I - -46. - - - - - - - - - I - - ---------------------------------------------------------------------------------------------------------- I ---------------- DDH------------------------------------------------------------------------ I I I TIME = 60.000H DDH * Diagnostic Var., AREA GLOBAL I I I I-------------------------------------------------------------------------I I VARIABLE I Initial value I Current value I I-------------------------------------------------------------------------I I OMEGA (P/S) I 0.1414511045E-21 I 0.1658986286E-06 I ICloud fractionI 0.0000000000E+00 I 0.5077553127E-03 I I Relat. Hum. I 0.4557032430E+00 I 0.4649002918E+00 I I PASS 01 I 0.0000000000E+00 I 0.5978551336E+01 I DDH------------------------------------------------------------------------ MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 240 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+000240 MODE=a IO-STREAM CLOSED - ICMGGh7zg+000240 MPI-TASK: 1 - 307745377 BYTES IN 7 RECORDS TRANSFERRED IN 0.0005 SECONDS615490.7 Mbytes/s, TOTAL TIME= 0.0108( 4.6%) 16:38:02 STEP 240 H= 60:00 +CPU= 31.448 NSTEP = 241 STEPO 0AAA00AAA 16:38:29 STEP 241 H= 60:15 +CPU= 21.911 NSTEP = 242 STEPO 0AAA00AAA 16:38:59 STEP 242 H= 60:30 +CPU= 23.995 NSTEP = 243 STEPO 0AAA00AAA 16:39:28 STEP 243 H= 60:45 +CPU= 23.456 NSTEP = 244 STEPO 0AAA00AAA 16:39:58 STEP 244 H= 61:00 +CPU= 23.605 ISTEP= 245 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482 ISTEP= 245 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00 NSTEP = 245 STEPO 0AAA00AAA 16:40:29 STEP 245 H= 61:15 +CPU= 24.753 NSTEP = 246 STEPO 0AAA00AAA 16:40:57 STEP 246 H= 61:30 +CPU= 21.659 NSTEP = 247 STEPO 0AAA00AAA 16:41:24 STEP 247 H= 61:45 +CPU= 22.846 NSTEP = 248 STEPO 0AAA00AAA 16:41:51 STEP 248 H= 62:00 +CPU= 23.263 NSTEP = 249 STEPO 0AAA00AAA ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 16:38:59 0AAA00AAA STEPO 243 23.995 23.995 30.159 97:24 -3431:** 0.99945229889660E-07 2GB 0MB 16:39:28 0AAA00AAA STEPO 244 23.456 23.456 29.562 97:48 -3430:** 0.99945229889660E-07 2GB 0MB 16:39:58 0AAA00AAA STEPO 245 23.605 23.605 29.683 98:11 -3430:** 0.99945229889660E-07 2GB 0MB 16:40:29 0AAA00AAA STEPO 246 24.753 24.753 30.984 98:36 -3429:** 0.99945229889660E-07 2GB 0MB 16:40:57 0AAA00AAA STEPO 247 21.660 21.660 27.793 98:58 -3429:** 0.99945229889660E-07 2GB 0MB 16:41:24 0AAA00AAA STEPO 248 22.846 22.846 26.909 99:20 -3429:** 0.99945229889660E-07 2GB 0MB 16:41:51 0AAA00AAA STEPO 249 23.263 23.263 27.342 99:44 -3428:** 0.99945229889660E-07 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 240", CTIME = "00021200 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 228, 2*0, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 0.000000000000000E+000, GMASSI = 0.000000000000000E+000 / ------------------------------------------------ >>> Printing last 8 lines from file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml <running_values> <last_cpu_time>6078.870000</last_cpu_time> <upload_file_number>2</upload_file_number> <last_iter>249</last_iter> <last_upload>172800</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully (argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu (argv1) start_date: 2016092300 (argv2) exptid: h7zg (argv3) unique_member_id: a0zu (argv4) batchid: 991 (argv5) wuid: 12208077 (argv6) fclen: 15 (argv7) app_name: oifs_43r3_bl (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/13 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_bl version: 1.11 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/13/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/13/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_d431967cdf9a07b90a7d41a8f727ae52 to: /var/lib/boinc-client/slots/13/oifs_43r3_bl_a0zu_2016092300_15_991_12208077.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/13/oifs_43r3_bl_a0zu_2016092300_15_991_12208077.zip ic_ancil_file: ic_ancil_12208077 ifsdata_file: ifsdata_12208077 climate_data_file: clim_data_12208077 horiz_resolution: 159 vert_resolution: 91 grid_type: l_2 upload_interval: 96 utstep: 900 nfrres: restart dump frequency (steps) 48 Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/13/ic_ancil_12208077.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/13/ic_ancil_12208077.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_74b8198107a41083cc31bed7a12ef4c5 to: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12208077.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/13/ifsdata/ifsdata_12208077.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/13/159l_2/clim_data_12208077.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/13/159l_2/clim_data_12208077.zip Checking for progress XML file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml Opened progress file ok : /var/lib/boinc-client/slots/13/progress_file_12208077.xml -- Model is restarting -- Adjusting last_iter, 249, to previous model restart step. Creating progress file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml last_cpu_time: 6078 upload_file_number: 2 last_iter: 241 last_upload: 172800 model_completed: 0 total_length_of_simulation: 1296000 result_base_name: oifs_43r3_bl_a0zu_2016092300_15_991_12208077_0_r634992926 The child process has been launched with process id: 601457 Executing the command: /var/lib/boinc-client/slots/13/oifs_43r3_model.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil)) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil)) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil)) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil)) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil)) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil)) [EC_DRHOOK:LM8:1:1:601457:601457] [20230213:164637:1676324797:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:47:11 STEP 240 H= 60:00 +CPU= 24.626 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:47:41 STEP 241 H= 60:15 +CPU= 23.496 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:48:11 STEP 242 H= 60:30 +CPU= 23.871 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:48:42 STEP 243 H= 60:45 +CPU= 24.575 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:49:12 STEP 244 H= 61:00 +CPU= 24.029 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:49:42 STEP 245 H= 61:15 +CPU= 23.048 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:50:13 STEP 246 H= 61:30 +CPU= 24.707 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:50:42 STEP 247 H= 61:45 +CPU= 22.680 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:51:12 STEP 248 H= 62:00 +CPU= 23.178 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:51:40 STEP 249 H= 62:15 +CPU= 22.213 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:52:09 STEP 250 H= 62:30 +CPU= 22.638 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:52:38 STEP 251 H= 62:45 +CPU= 23.189 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:53:15 STEP 252 H= 63:00 +CPU= 29.592 Suspend request received from the BOINC client, suspending the child process Resuming the child process Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000252 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000252 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000252 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:53:47 STEP 253 H= 63:15 +CPU= 24.098 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:54:17 STEP 254 H= 63:30 +CPU= 24.285 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:54:47 STEP 255 H= 63:45 +CPU= 23.816 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:55:19 STEP 256 H= 64:00 +CPU= 25.659 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:55:49 STEP 257 H= 64:15 +CPU= 23.460 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:56:19 STEP 258 H= 64:30 +CPU= 23.969 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:56:47 STEP 259 H= 64:45 +CPU= 24.370 Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:57:12 STEP 260 H= 65:00 +CPU= 22.790 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:57:42 STEP 261 H= 65:15 +CPU= 23.475 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:58:14 STEP 262 H= 65:30 +CPU= 25.206 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:58:43 STEP 263 H= 65:45 +CPU= 24.859 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:59:22 STEP 264 H= 66:00 +CPU= 31.621 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000264 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000264 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000264 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 16:59:57 STEP 265 H= 66:15 +CPU= 26.968 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:00:28 STEP 266 H= 66:30 +CPU= 24.623 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:02:09 STEP 267 H= 66:45 +CPU= 24.851 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:02:37 STEP 268 H= 67:00 +CPU= 22.684 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:03:07 STEP 269 H= 67:15 +CPU= 23.514 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:19:15 STEP 270 H= 67:30 +CPU= 25.586 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:19:43 STEP 271 H= 67:45 +CPU= 21.883 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:20:10 STEP 272 H= 68:00 +CPU= 22.394 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:30:45 STEP 273 H= 68:15 +CPU= 27.034 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:31:15 STEP 274 H= 68:30 +CPU= 23.408 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:31:42 STEP 275 H= 68:45 +CPU= 23.084 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:32:46 STEP 276 H= 69:00 +CPU= 29.656 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000276 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000276 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000276 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:33:12 STEP 277 H= 69:15 +CPU= 21.914 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:33:40 STEP 278 H= 69:30 +CPU= 23.678 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:34:09 STEP 279 H= 69:45 +CPU= 24.451 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:35:05 STEP 280 H= 70:00 +CPU= 26.405 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:37:26 STEP 281 H= 70:15 +CPU= 26.325 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:37:56 STEP 282 H= 70:30 +CPU= 23.683 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:38:27 STEP 283 H= 70:45 +CPU= 25.119 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:39:16 STEP 284 H= 71:00 +CPU= 24.213 Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:39:40 STEP 285 H= 71:15 +CPU= 22.004 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:40:13 STEP 286 H= 71:30 +CPU= 24.844 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:40:43 STEP 287 H= 71:45 +CPU= 24.084 Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process Suspend request received from the BOINC client, suspending the child process Resuming the child process 17:41:25 STEP 288 H= 72:00 +CPU= 32.065 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMGGh7zg+000288 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMSHh7zg+000288 Moving to projects directory: /var/lib/boinc-client/slots/13/ICMUAh7zg+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMGGh7zg+000276 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMSHh7zg+000276 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12208077/ICMUAh7zg+000276 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a0zu_2016092300_15_991_12208077_0_r634992926_2.zip Uploading the intermediate file: upload_file_2.zip 17:41:47 STEP 289 H= 72:15 +CPU= 22.220 Suspend request received from the BOINC client, suspending the child process Resuming the child process ..The child process has been killed with signal: 9 >>> Printing last 70 lines from file: NODE.001_01 GPNORM RAIN AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CLOUD FRACTION AVERAGE MINIMUM MAXIMUM AVE 0.104295293200559E-01 0.000000000000000E+00 0.100000000000000E+01 GPNORM CLW AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CIW AVERAGE MINIMUM MAXIMUM AVE 0.576137522434097E-09 0.000000000000000E+00 0.526710797418912E-07 GPNORM CC AVERAGE MINIMUM MAXIMUM AVE 0.104677350890231E-01 0.000000000000000E+00 0.100000000000000E+01 GPNORM RFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM SFL AVERAGE MINIMUM MAXIMUM AVE 0.594646321555523E-12 0.000000000000000E+00 0.203317735571648E-10 GPNORM CRFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM CSFL AVERAGE MINIMUM MAXIMUM AVE 0.000000000000000E+00 0.000000000000000E+00 0.000000000000000E+00 GPNORM PFRC AVERAGE MINIMUM MAXIMUM AVE 0.791136656756934E-02 0.000000000000000E+00 0.100000000000000E+01 NSTEP = 288 STEPO F00000000 CREATE RESTART FILE ./srf00030000.0001 IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./srf00030000.0001 MODE=w IO-STREAM CLOSED - ./srf00030000.0001 MPI-TASK: 1 - ************ BYTES IN 34 RECORDS TRANSFERRED IN 2.1279 SECONDS******** Mbytes/s, TOTAL TIME= 3.4839(61.1%) CREATE RESTART CONTROL FILE ./rcf WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00021200.0001 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 288 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+000288 MODE=w IO-STREAM CLOSED - ICMGGh7zg+000288 MPI-TASK: 1 - 311589210 BYTES IN 6 RECORDS TRANSFERRED IN 0.0003 SECONDS******** Mbytes/s, TOTAL TIME= 0.0884( 0.3%) NSTEP = 288 STEPO 0AA000000 IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMSHh7zg+000288 MODE=a IO-STREAM CLOSED - ./ICMSHh7zg+000288 MPI-TASK: 1 - 707765250 BYTES IN 202 RECORDS TRANSFERRED IN 0.0098 SECONDS 72220.9 Mbytes/s, TOTAL TIME= 0.4079( 2.4%) IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000288 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000288 MPI-TASK: 1 - 4985427351 BYTES IN 125 RECORDS TRANSFERRED IN 0.0023 SECONDS******** Mbytes/s, TOTAL TIME= 0.1219( 1.9%) STEPO_FPOS WAS : B00000F00E IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000288 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000288 MPI-TASK: 1 - 303015577 BYTES IN 2 RECORDS TRANSFERRED IN 0.0001 SECONDS******** Mbytes/s, TOTAL TIME= 0.0077( 1.3%) STEPO_FPOS WAS : V00000000Y IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ./ICMUAh7zg+000288 MODE=a IO-STREAM CLOSED - ./ICMUAh7zg+000288 MPI-TASK: 1 - 311589210 BYTES IN 4 RECORDS TRANSFERRED IN 0.0002 SECONDS******** Mbytes/s, TOTAL TIME= 0.0120( 1.7%) STEPO_FPOS WAS : T00000000M NSTEP = 288 STEPO 0AAA00AAA MAXGPFV : MAX. VALUE = 0.000000000000000E+000 MAXGPFV : MAX. VALUE = 0.000000000000000E+000 NSTEP = 288 SCAN2M_HPOS P IO-STREAM SETUP - IOTYPE = 2 NUMIOPROCS = 1 CPATH = ICMGGh7zg+000288 MODE=a IO-STREAM CLOSED - ICMGGh7zg+000288 MPI-TASK: 1 - 311589201 BYTES IN 7 RECORDS TRANSFERRED IN 0.0004 SECONDS778973.0 Mbytes/s, TOTAL TIME= 0.0103( 3.9%) 17:41:25 STEP 288 H= 72:00 +CPU= 32.065 NSTEP = 289 STEPO 0AAA00AAA 17:41:47 STEP 289 H= 72:15 +CPU= 22.220 ISTEP= 290 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482 ISTEP= 290 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00 NSTEP = 290 STEPO 0AAA00AAA ------------------------------------------------ CNT0 not found; string returned was: 'STEPO' >>> Printing last 8 lines from file: ifs.stat 17:40:48 0AA000000 STEPO 288 3.227 3.227 3.971 19:39 54:10 0.10830779545655E-06 2GB 0MB 17:40:49 FULLPOS-B DYNFPOS 288 1.383 1.383 1.395 19:40 54:12 0.10830779545655E-06 2GB 0MB 17:40:54 FULLPOS-V DYNFPOS 288 2.559 2.559 4.788 19:42 54:16 0.10830779545655E-06 2GB 0MB 17:40:54 FULLPOS-T DYNFPOS 288 0.476 0.476 0.494 19:43 54:17 0.10830779545655E-06 2GB 0MB 17:40:55 0AAA00AAA STEPO 288 0.447 0.447 0.454 19:43 54:17 0.10830779545655E-06 2GB 0MB 17:41:25 0AAA00AAA STEPO 289 23.774 23.774 29.910 20:07 54:47 0.10830779545655E-06 2GB 0MB 17:41:47 0AAA00AAA STEPO 290 22.221 22.221 22.875 20:29 55:10 0.10830779545655E-06 2GB 0MB ------------------------------------------------ >>> Printing last 11 lines from file: rcf CSTEP = " 288", CTIME = "00030000 ", NSTEPLPP = 201, 202, 49, 228226, 228227, 2*0, 276, 2*0, IPRGPNSRES = 1, IPRGPEWRES = 1, IPRTRWRES = 1, IPRTRVRES = 1, GMASS0 = 0.000000000000000E+000, GMASSI = 0.000000000000000E+000 / ------------------------------------------------ >>> Printing last 8 lines from file: /var/lib/boinc-client/slots/13/progress_file_12208077.xml <running_values> <last_cpu_time>7286.080000</last_cpu_time> <upload_file_number>3</upload_file_number> <last_iter>289</last_iter> <last_upload>259200</last_upload> <model_completed>0</model_completed> </running_values> ------------------------------------------------ ..Failed, model did not complete successfully </stderr_txt> ]]> |
No trickles! |
---|
©2024 climateprediction.net