Name | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 |
Workunit | 12163774 |
Created | 28 Nov 2022, 20:56:35 UTC |
Sent | 28 Nov 2022, 23:21:10 UTC |
Report deadline | 28 Dec 2022, 23:21:10 UTC |
Received | 13 Dec 2022, 7:55:02 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 1 (0x00000001) Unknown error code |
Computer ID | 1522983 |
Run time | 8 hours 57 min 6 sec |
CPU time | 4 hours 15 min 25 sec |
Validate state | Invalid |
Credit | 4,596.48 |
Device peak FLOPS | 5.38 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.01 x86_64-pc-linux-gnu |
Peak working set size | 4,219.65 MB |
Peak swap size | 4,978.05 MB |
Peak disk usage | 2,297.92 MB |
Stderr | <core_client_version>7.16.6</core_client_version> <![CDATA[ <message> process exited with code 1 (0x1, -255)</message> <stderr_txt> :36:23 STEP 394 H= 394:00 +CPU= 29.941 03:36:50 STEP 395 H= 395:00 +CPU= 27.153 03:37:35 STEP 396 H= 396:00 +CPU= 45.166 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000396 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000396 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000396 03:38:03 STEP 397 H= 397:00 +CPU= 27.712 03:38:32 STEP 398 H= 398:00 +CPU= 28.923 03:39:14 STEP 399 H= 399:00 +CPU= 42.355 03:39:42 STEP 400 H= 400:00 +CPU= 27.957 03:40:09 STEP 401 H= 401:00 +CPU= 26.769 03:40:52 STEP 402 H= 402:00 +CPU= 43.020 03:41:21 STEP 403 H= 403:00 +CPU= 28.257 03:41:50 STEP 404 H= 404:00 +CPU= 29.467 03:42:34 STEP 405 H= 405:00 +CPU= 44.113 03:43:02 STEP 406 H= 406:00 +CPU= 27.526 03:43:30 STEP 407 H= 407:00 +CPU= 28.480 03:44:19 STEP 408 H= 408:00 +CPU= 47.963 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000408 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000408 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000396 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_16.zip Uploading the intermediate file: upload_file_16.zip Uploading trickle at timestep: 1468800 03:44:46 STEP 409 H= 409:00 +CPU= 27.042 03:45:14 STEP 410 H= 410:00 +CPU= 27.997 03:45:55 STEP 411 H= 411:00 +CPU= 41.753 03:46:24 STEP 412 H= 412:00 +CPU= 28.958 03:46:52 STEP 413 H= 413:00 +CPU= 27.546 03:47:35 STEP 414 H= 414:00 +CPU= 42.986 03:48:04 STEP 415 H= 415:00 +CPU= 29.049 03:48:33 STEP 416 H= 416:00 +CPU= 28.480 03:49:13 STEP 417 H= 417:00 +CPU= 40.185 03:49:40 STEP 418 H= 418:00 +CPU= 27.262 03:50:08 STEP 419 H= 419:00 +CPU= 28.029 03:50:53 STEP 420 H= 420:00 +CPU= 44.100 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000420 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000420 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000420 03:51:21 STEP 421 H= 421:00 +CPU= 28.049 03:51:50 STEP 422 H= 422:00 +CPU= 29.689 03:52:30 STEP 423 H= 423:00 +CPU= 39.768 03:53:01 STEP 424 H= 424:00 +CPU= 30.246 03:53:28 STEP 425 H= 425:00 +CPU= 27.121 03:54:13 STEP 426 H= 426:00 +CPU= 45.099 03:54:41 STEP 427 H= 427:00 +CPU= 28.141 03:55:10 STEP 428 H= 428:00 +CPU= 28.554 03:55:51 STEP 429 H= 429:00 +CPU= 40.896 03:56:19 STEP 430 H= 430:00 +CPU= 28.531 03:56:50 STEP 431 H= 431:00 +CPU= 30.303 03:57:38 STEP 432 H= 432:00 +CPU= 48.647 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000432 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000432 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000420 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_17.zip Uploading the intermediate file: upload_file_17.zip Uploading trickle at timestep: 1555200 03:58:07 STEP 433 H= 433:00 +CPU= 28.135 03:58:34 STEP 434 H= 434:00 +CPU= 27.452 03:59:17 STEP 435 H= 435:00 +CPU= 42.737 03:59:44 STEP 436 H= 436:00 +CPU= 27.157 04:00:12 STEP 437 H= 437:00 +CPU= 27.914 04:00:55 STEP 438 H= 438:00 +CPU= 42.630 04:01:22 STEP 439 H= 439:00 +CPU= 27.373 04:01:52 STEP 440 H= 440:00 +CPU= 29.126 04:02:34 STEP 441 H= 441:00 +CPU= 42.496 04:03:03 STEP 442 H= 442:00 +CPU= 28.817 04:03:31 STEP 443 H= 443:00 +CPU= 28.161 04:04:18 STEP 444 H= 444:00 +CPU= 46.443 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000444 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000444 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000444 04:04:45 STEP 445 H= 445:00 +CPU= 26.935 04:05:14 STEP 446 H= 446:00 +CPU= 29.281 04:05:55 STEP 447 H= 447:00 +CPU= 41.343 04:06:26 STEP 448 H= 448:00 +CPU= 29.967 04:06:53 STEP 449 H= 449:00 +CPU= 27.462 04:07:36 STEP 450 H= 450:00 +CPU= 43.054 04:08:04 STEP 451 H= 451:00 +CPU= 27.560 04:08:34 STEP 452 H= 452:00 +CPU= 30.005 04:09:16 STEP 453 H= 453:00 +CPU= 42.171 04:09:44 STEP 454 H= 454:00 +CPU= 28.078 04:10:13 STEP 455 H= 455:00 +CPU= 28.484 04:11:03 STEP 456 H= 456:00 +CPU= 50.277 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000456 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000456 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000444 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_18.zip Uploading the intermediate file: upload_file_18.zip Uploading trickle at timestep: 1641600 04:11:30 STEP 457 H= 457:00 +CPU= 26.458 04:11:57 STEP 458 H= 458:00 +CPU= 27.252 04:12:41 STEP 459 H= 459:00 +CPU= 43.372 04:13:09 STEP 460 H= 460:00 +CPU= 28.149 04:13:38 STEP 461 H= 461:00 +CPU= 28.873 04:14:21 STEP 462 H= 462:00 +CPU= 42.885 04:14:48 STEP 463 H= 463:00 +CPU= 27.131 04:15:18 STEP 464 H= 464:00 +CPU= 29.936 04:16:00 STEP 465 H= 465:00 +CPU= 41.651 04:16:28 STEP 466 H= 466:00 +CPU= 28.186 04:16:56 STEP 467 H= 467:00 +CPU= 27.797 04:17:43 STEP 468 H= 468:00 +CPU= 47.199 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000468 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000468 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000468 04:18:09 STEP 469 H= 469:00 +CPU= 26.044 04:18:38 STEP 470 H= 470:00 +CPU= 29.219 04:19:21 STEP 471 H= 471:00 +CPU= 42.663 04:19:50 STEP 472 H= 472:00 +CPU= 28.488 04:20:18 STEP 473 H= 473:00 +CPU= 28.030 04:20:59 STEP 474 H= 474:00 +CPU= 41.501 04:21:27 STEP 475 H= 475:00 +CPU= 27.769 04:21:57 STEP 476 H= 476:00 +CPU= 29.307 04:22:41 STEP 477 H= 477:00 +CPU= 44.061 04:23:08 STEP 478 H= 478:00 +CPU= 27.645 04:23:40 STEP 479 H= 479:00 +CPU= 31.375 04:24:29 STEP 480 H= 480:00 +CPU= 48.759 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000480 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000480 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000468 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000468 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000468 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_19.zip Uploading the intermediate file: upload_file_19.zip 04:24:57 STEP 481 H= 481:00 +CPU= 27.769 Uploading trickle at timestep: 1728000 04:25:24 STEP 482 H= 482:00 +CPU= 27.597 04:26:06 STEP 483 H= 483:00 +CPU= 41.249 04:26:35 STEP 484 H= 484:00 +CPU= 29.065 04:27:02 STEP 485 H= 485:00 +CPU= 27.251 04:27:48 STEP 486 H= 486:00 +CPU= 45.468 04:28:14 STEP 487 H= 487:00 +CPU= 26.835 04:28:43 STEP 488 H= 488:00 +CPU= 28.635 04:29:25 STEP 489 H= 489:00 +CPU= 41.830 04:29:54 STEP 490 H= 490:00 +CPU= 29.117 04:30:23 STEP 491 H= 491:00 +CPU= 28.361 04:31:09 STEP 492 H= 492:00 +CPU= 46.541 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000492 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000492 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000492 04:31:36 STEP 493 H= 493:00 +CPU= 26.899 Quit request received from BOINC client, ending the child process (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu (argv1) start_date: 2021050100 (argv2) exptid: hpi1 (argv3) unique_member_id: 0685 (argv4) batchid: 945 (argv5) wuid: 12163774 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/8 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.01 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_8b6eabf9b8e55442bbc3f53368b7cb1f to: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0685_2021050100_123_945_12163774.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_ps_0685_2021050100_123_945_12163774.zip ic_ancil_file: ic_ancil_12163774 ifsdata_file: ifsdata_12163774 climate_data_file: clim_data_12163774 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 Copying IC ancils from: ../../projects/climateprediction.net/jf_b0d3c690ccc453bb4dc076441c4d9637 to: /var/lib/boinc-client/slots/8/ic_ancil_12163774.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12163774.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e6e57fe9b8829d8ceb6dfabee525f7d8 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12163774.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12163774.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12163774.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12163774.zip total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209 The child process has been launched with process id: 969154 Executing the command: /var/lib/boinc-client/slots/8/./master.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:B30:1:1:969154:969154] [20221213:043306:1670902386:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 04:33:58 STEP 480 H= 480:00 +CPU= 43.631 04:34:24 STEP 481 H= 481:00 +CPU= 26.313 04:34:54 STEP 482 H= 482:00 +CPU= 28.600 04:35:38 STEP 483 H= 483:00 +CPU= 44.358 04:36:06 STEP 484 H= 484:00 +CPU= 27.613 04:36:34 STEP 485 H= 485:00 +CPU= 27.654 04:37:17 STEP 486 H= 486:00 +CPU= 43.064 04:37:44 STEP 487 H= 487:00 +CPU= 27.031 04:38:13 STEP 488 H= 488:00 +CPU= 28.336 04:38:56 STEP 489 H= 489:00 +CPU= 43.287 04:39:23 STEP 490 H= 490:00 +CPU= 26.962 04:39:51 STEP 491 H= 491:00 +CPU= 27.688 04:40:37 STEP 492 H= 492:00 +CPU= 45.981 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000492 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000492 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000492 04:41:05 STEP 493 H= 493:00 +CPU= 28.099 04:41:34 STEP 494 H= 494:00 +CPU= 29.085 04:42:16 STEP 495 H= 495:00 +CPU= 41.463 04:42:43 STEP 496 H= 496:00 +CPU= 27.609 04:43:11 STEP 497 H= 497:00 +CPU= 27.345 04:43:55 STEP 498 H= 498:00 +CPU= 44.169 04:44:22 STEP 499 H= 499:00 +CPU= 27.055 04:44:51 STEP 500 H= 500:00 +CPU= 29.023 04:45:35 STEP 501 H= 501:00 +CPU= 44.224 04:46:03 STEP 502 H= 502:00 +CPU= 27.981 04:46:31 STEP 503 H= 503:00 +CPU= 27.635 04:47:23 STEP 504 H= 504:00 +CPU= 49.423 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000504 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000504 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000492 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000492 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000492 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_1.zip Uploading the intermediate file: upload_file_1.zip 04:47:50 STEP 505 H= 505:00 +CPU= 27.078 Uploading trickle at timestep: 1814400 04:48:18 STEP 506 H= 506:00 +CPU= 28.390 04:49:02 STEP 507 H= 507:00 +CPU= 43.582 04:49:30 STEP 508 H= 508:00 +CPU= 28.000 04:49:58 STEP 509 H= 509:00 +CPU= 27.255 04:50:42 STEP 510 H= 510:00 +CPU= 44.387 04:51:11 STEP 511 H= 511:00 +CPU= 28.909 04:51:39 STEP 512 H= 512:00 +CPU= 28.135 04:52:22 STEP 513 H= 513:00 +CPU= 42.950 04:52:51 STEP 514 H= 514:00 +CPU= 28.903 04:53:19 STEP 515 H= 515:00 +CPU= 27.405 04:54:07 STEP 516 H= 516:00 +CPU= 47.999 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000516 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000516 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000516 04:54:34 STEP 517 H= 517:00 +CPU= 27.360 04:55:02 STEP 518 H= 518:00 +CPU= 27.919 04:55:45 STEP 519 H= 519:00 +CPU= 42.648 04:56:15 STEP 520 H= 520:00 +CPU= 29.676 04:56:43 STEP 521 H= 521:00 +CPU= 27.695 04:57:26 STEP 522 H= 522:00 +CPU= 43.529 04:57:55 STEP 523 H= 523:00 +CPU= 29.017 04:58:23 STEP 524 H= 524:00 +CPU= 27.799 04:59:06 STEP 525 H= 525:00 +CPU= 43.168 04:59:36 STEP 526 H= 526:00 +CPU= 29.817 05:00:04 STEP 527 H= 527:00 +CPU= 27.961 05:00:55 STEP 528 H= 528:00 +CPU= 50.864 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000528 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000528 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000516 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000516 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000516 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_2.zip Uploading the intermediate file: upload_file_2.zip Uploading trickle at timestep: 1900800 05:01:25 STEP 529 H= 529:00 +CPU= 29.248 05:01:53 STEP 530 H= 530:00 +CPU= 28.573 05:02:37 STEP 531 H= 531:00 +CPU= 43.533 05:03:06 STEP 532 H= 532:00 +CPU= 28.706 05:03:33 STEP 533 H= 533:00 +CPU= 27.178 05:04:18 STEP 534 H= 534:00 +CPU= 44.988 05:04:46 STEP 535 H= 535:00 +CPU= 28.204 05:05:14 STEP 536 H= 536:00 +CPU= 27.922 05:05:57 STEP 537 H= 537:00 +CPU= 42.827 05:06:26 STEP 538 H= 538:00 +CPU= 29.234 05:06:54 STEP 539 H= 539:00 +CPU= 27.931 05:07:41 STEP 540 H= 540:00 +CPU= 46.452 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000540 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000540 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000540 05:08:07 STEP 541 H= 541:00 +CPU= 26.151 05:08:36 STEP 542 H= 542:00 +CPU= 29.110 05:09:20 STEP 543 H= 543:00 +CPU= 43.639 05:09:48 STEP 544 H= 544:00 +CPU= 28.041 05:10:14 STEP 545 H= 545:00 +CPU= 25.312 05:10:57 STEP 546 H= 546:00 +CPU= 43.375 05:11:27 STEP 547 H= 547:00 +CPU= 29.452 05:11:55 STEP 548 H= 548:00 +CPU= 27.980 05:12:37 STEP 549 H= 549:00 +CPU= 42.210 05:13:07 STEP 550 H= 550:00 +CPU= 29.699 05:13:35 STEP 551 H= 551:00 +CPU= 28.740 05:14:26 STEP 552 H= 552:00 +CPU= 50.572 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000552 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000552 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000540 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000540 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000540 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_3.zip Uploading the intermediate file: upload_file_3.zip 05:14:54 STEP 553 H= 553:00 +CPU= 27.690 Uploading trickle at timestep: 1987200 05:15:22 STEP 554 H= 554:00 +CPU= 27.754 05:16:04 STEP 555 H= 555:00 +CPU= 42.506 05:16:34 STEP 556 H= 556:00 +CPU= 29.766 05:17:01 STEP 557 H= 557:00 +CPU= 26.580 05:17:45 STEP 558 H= 558:00 +CPU= 44.070 05:18:13 STEP 559 H= 559:00 +CPU= 27.999 05:18:42 STEP 560 H= 560:00 +CPU= 28.560 05:19:23 STEP 561 H= 561:00 +CPU= 41.247 05:19:52 STEP 562 H= 562:00 +CPU= 28.894 05:20:19 STEP 563 H= 563:00 +CPU= 27.453 05:21:07 STEP 564 H= 564:00 +CPU= 47.133 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000564 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000564 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000564 05:21:36 STEP 565 H= 565:00 +CPU= 28.966 05:22:03 STEP 566 H= 566:00 +CPU= 27.662 05:22:47 STEP 567 H= 567:00 +CPU= 43.319 05:23:16 STEP 568 H= 568:00 +CPU= 29.512 05:23:43 STEP 569 H= 569:00 +CPU= 26.865 05:24:25 STEP 570 H= 570:00 +CPU= 42.018 05:24:56 STEP 571 H= 571:00 +CPU= 30.070 05:25:23 STEP 572 H= 572:00 +CPU= 27.525 05:26:05 STEP 573 H= 573:00 +CPU= 42.095 05:26:36 STEP 574 H= 574:00 +CPU= 30.924 05:27:05 STEP 575 H= 575:00 +CPU= 28.984 05:27:58 STEP 576 H= 576:00 +CPU= 52.683 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000576 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000576 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000564 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000564 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000564 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_4.zip Uploading the intermediate file: upload_file_4.zip 05:28:25 STEP 577 H= 577:00 +CPU= 27.121 Uploading trickle at timestep: 2073600 05:28:53 STEP 578 H= 578:00 +CPU= 28.011 05:29:37 STEP 579 H= 579:00 +CPU= 43.737 05:30:07 STEP 580 H= 580:00 +CPU= 29.286 05:30:35 STEP 581 H= 581:00 +CPU= 28.486 05:31:18 STEP 582 H= 582:00 +CPU= 42.485 05:31:46 STEP 583 H= 583:00 +CPU= 28.003 05:32:14 STEP 584 H= 584:00 +CPU= 28.468 05:32:56 STEP 585 H= 585:00 +CPU= 41.617 05:33:24 STEP 586 H= 586:00 +CPU= 28.021 05:33:50 STEP 587 H= 587:00 +CPU= 25.921 05:34:38 STEP 588 H= 588:00 +CPU= 48.239 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000588 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000588 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000588 05:35:07 STEP 589 H= 589:00 +CPU= 28.760 05:35:36 STEP 590 H= 590:00 +CPU= 28.535 05:36:18 STEP 591 H= 591:00 +CPU= 41.776 05:36:47 STEP 592 H= 592:00 +CPU= 28.829 05:37:15 STEP 593 H= 593:00 +CPU= 27.877 05:37:58 STEP 594 H= 594:00 +CPU= 43.211 05:38:25 STEP 595 H= 595:00 +CPU= 27.312 05:38:53 STEP 596 H= 596:00 +CPU= 27.483 05:39:36 STEP 597 H= 597:00 +CPU= 43.656 05:40:06 STEP 598 H= 598:00 +CPU= 29.839 05:40:35 STEP 599 H= 599:00 +CPU= 29.060 05:41:26 STEP 600 H= 600:00 +CPU= 50.071 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000600 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000600 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000588 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000588 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000588 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_5.zip Uploading the intermediate file: upload_file_5.zip Uploading trickle at timestep: 2160000 05:41:53 STEP 601 H= 601:00 +CPU= 27.550 05:42:23 STEP 602 H= 602:00 +CPU= 29.914 05:43:05 STEP 603 H= 603:00 +CPU= 42.074 05:43:34 STEP 604 H= 604:00 +CPU= 28.740 05:44:01 STEP 605 H= 605:00 +CPU= 27.127 05:44:45 STEP 606 H= 606:00 +CPU= 43.857 05:45:13 STEP 607 H= 607:00 +CPU= 27.313 05:45:41 STEP 608 H= 608:00 +CPU= 28.180 05:46:22 STEP 609 H= 609:00 +CPU= 41.470 05:46:51 STEP 610 H= 610:00 +CPU= 28.459 05:47:19 STEP 611 H= 611:00 +CPU= 28.030 05:48:04 STEP 612 H= 612:00 +CPU= 44.980 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000612 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000612 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000612 05:48:33 STEP 613 H= 613:00 +CPU= 28.358 05:49:02 STEP 614 H= 614:00 +CPU= 29.230 05:49:44 STEP 615 H= 615:00 +CPU= 42.088 05:50:14 STEP 616 H= 616:00 +CPU= 29.716 05:50:43 STEP 617 H= 617:00 +CPU= 29.049 05:51:26 STEP 618 H= 618:00 +CPU= 43.065 05:51:55 STEP 619 H= 619:00 +CPU= 29.074 05:52:24 STEP 620 H= 620:00 +CPU= 29.246 05:53:07 STEP 621 H= 621:00 +CPU= 42.409 05:53:36 STEP 622 H= 622:00 +CPU= 28.660 05:54:04 STEP 623 H= 623:00 +CPU= 28.638 05:54:56 STEP 624 H= 624:00 +CPU= 51.603 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000624 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000624 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000624 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000612 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000612 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000612 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_6.zip Uploading the intermediate file: upload_file_6.zip Uploading trickle at timestep: 2246400 05:55:25 STEP 625 H= 625:00 +CPU= 29.273 05:55:54 STEP 626 H= 626:00 +CPU= 28.181 05:56:36 STEP 627 H= 627:00 +CPU= 41.877 05:57:04 STEP 628 H= 628:00 +CPU= 28.272 05:57:32 STEP 629 H= 629:00 +CPU= 28.388 05:58:16 STEP 630 H= 630:00 +CPU= 43.774 05:58:44 STEP 631 H= 631:00 +CPU= 27.546 05:59:12 STEP 632 H= 632:00 +CPU= 28.433 05:59:53 STEP 633 H= 633:00 +CPU= 40.454 06:00:23 STEP 634 H= 634:00 +CPU= 29.547 06:00:52 STEP 635 H= 635:00 +CPU= 29.287 06:01:37 STEP 636 H= 636:00 +CPU= 45.418 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000636 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000636 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000636 06:02:05 STEP 637 H= 637:00 +CPU= 27.648 06:02:35 STEP 638 H= 638:00 +CPU= 29.438 06:03:17 STEP 639 H= 639:00 +CPU= 42.344 06:03:45 STEP 640 H= 640:00 +CPU= 28.136 06:04:14 STEP 641 H= 641:00 +CPU= 28.554 06:04:57 STEP 642 H= 642:00 +CPU= 43.282 06:05:25 STEP 643 H= 643:00 +CPU= 27.789 06:05:55 STEP 644 H= 644:00 +CPU= 30.152 06:06:37 STEP 645 H= 645:00 +CPU= 41.796 06:07:06 STEP 646 H= 646:00 +CPU= 28.253 06:07:36 STEP 647 H= 647:00 +CPU= 29.958 06:08:26 STEP 648 H= 648:00 +CPU= 50.083 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000648 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000648 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000648 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000624 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000624 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000624 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000636 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000636 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000636 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_7.zip Uploading the intermediate file: upload_file_7.zip Uploading trickle at timestep: 2332800 06:08:54 STEP 649 H= 649:00 +CPU= 28.420 06:09:21 STEP 650 H= 650:00 +CPU= 26.605 06:10:02 STEP 651 H= 651:00 +CPU= 40.989 06:10:31 STEP 652 H= 652:00 +CPU= 29.227 06:10:59 STEP 653 H= 653:00 +CPU= 27.837 06:11:42 STEP 654 H= 654:00 +CPU= 42.701 06:12:10 STEP 655 H= 655:00 +CPU= 27.443 06:12:37 STEP 656 H= 656:00 +CPU= 27.783 06:13:22 STEP 657 H= 657:00 +CPU= 44.228 06:13:49 STEP 658 H= 658:00 +CPU= 27.159 06:14:16 STEP 659 H= 659:00 +CPU= 27.441 06:15:05 STEP 660 H= 660:00 +CPU= 48.260 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000660 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000660 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000660 06:15:34 STEP 661 H= 661:00 +CPU= 28.638 06:16:03 STEP 662 H= 662:00 +CPU= 29.784 06:16:44 STEP 663 H= 663:00 +CPU= 40.871 06:17:14 STEP 664 H= 664:00 +CPU= 29.668 06:17:42 STEP 665 H= 665:00 +CPU= 27.945 06:18:27 STEP 666 H= 666:00 +CPU= 44.966 06:18:55 STEP 667 H= 667:00 +CPU= 27.879 06:19:23 STEP 668 H= 668:00 +CPU= 28.212 06:20:04 STEP 669 H= 669:00 +CPU= 40.852 06:20:35 STEP 670 H= 670:00 +CPU= 30.123 06:21:03 STEP 671 H= 671:00 +CPU= 28.527 06:21:53 STEP 672 H= 672:00 +CPU= 50.082 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000672 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000672 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000672 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000648 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000648 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000648 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000660 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000660 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000660 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_8.zip Uploading the intermediate file: upload_file_8.zip Uploading trickle at timestep: 2419200 06:22:21 STEP 673 H= 673:00 +CPU= 27.902 06:22:49 STEP 674 H= 674:00 +CPU= 27.375 06:23:31 STEP 675 H= 675:00 +CPU= 42.319 06:23:59 STEP 676 H= 676:00 +CPU= 27.996 06:24:27 STEP 677 H= 677:00 +CPU= 28.032 06:25:08 STEP 678 H= 678:00 +CPU= 40.946 06:25:37 STEP 679 H= 679:00 +CPU= 28.290 06:26:05 STEP 680 H= 680:00 +CPU= 28.363 06:26:48 STEP 681 H= 681:00 +CPU= 42.796 06:27:17 STEP 682 H= 682:00 +CPU= 28.672 06:27:44 STEP 683 H= 683:00 +CPU= 27.126 06:28:31 STEP 684 H= 684:00 +CPU= 46.881 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000684 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000684 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000684 06:28:59 STEP 685 H= 685:00 +CPU= 28.274 06:29:28 STEP 686 H= 686:00 +CPU= 28.907 06:30:09 STEP 687 H= 687:00 +CPU= 40.506 06:30:37 STEP 688 H= 688:00 +CPU= 28.436 06:31:05 STEP 689 H= 689:00 +CPU= 28.122 06:31:50 STEP 690 H= 690:00 +CPU= 44.979 06:32:18 STEP 691 H= 691:00 +CPU= 27.937 06:32:48 STEP 692 H= 692:00 +CPU= 29.373 06:33:31 STEP 693 H= 693:00 +CPU= 42.677 06:33:58 STEP 694 H= 694:00 +CPU= 27.734 06:34:27 STEP 695 H= 695:00 +CPU= 28.153 06:35:16 STEP 696 H= 696:00 +CPU= 49.691 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000696 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000696 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000696 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000672 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000672 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000672 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000684 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000684 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000684 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_9.zip Uploading the intermediate file: upload_file_9.zip Uploading trickle at timestep: 2505600 06:35:45 STEP 697 H= 697:00 +CPU= 28.730 06:36:14 STEP 698 H= 698:00 +CPU= 29.174 06:36:57 STEP 699 H= 699:00 +CPU= 42.681 06:37:26 STEP 700 H= 700:00 +CPU= 28.400 06:37:53 STEP 701 H= 701:00 +CPU= 27.613 06:38:37 STEP 702 H= 702:00 +CPU= 43.192 06:39:06 STEP 703 H= 703:00 +CPU= 29.004 06:39:35 STEP 704 H= 704:00 +CPU= 28.938 06:40:18 STEP 705 H= 705:00 +CPU= 43.659 06:40:48 STEP 706 H= 706:00 +CPU= 29.213 06:41:15 STEP 707 H= 707:00 +CPU= 27.593 06:42:02 STEP 708 H= 708:00 +CPU= 46.669 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000708 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000708 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000708 06:42:30 STEP 709 H= 709:00 +CPU= 28.138 06:42:58 STEP 710 H= 710:00 +CPU= 27.355 06:43:39 STEP 711 H= 711:00 +CPU= 41.617 06:44:09 STEP 712 H= 712:00 +CPU= 29.782 06:44:37 STEP 713 H= 713:00 +CPU= 27.185 06:45:19 STEP 714 H= 714:00 +CPU= 42.736 06:45:47 STEP 715 H= 715:00 +CPU= 27.199 06:46:14 STEP 716 H= 716:00 +CPU= 27.715 06:46:58 STEP 717 H= 717:00 +CPU= 43.574 06:47:26 STEP 718 H= 718:00 +CPU= 28.106 06:47:55 STEP 719 H= 719:00 +CPU= 29.213 06:48:46 STEP 720 H= 720:00 +CPU= 50.629 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000720 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000720 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000720 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000696 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000696 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000696 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000708 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000708 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000708 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_10.zip Uploading the intermediate file: upload_file_10.zip 06:49:14 STEP 721 H= 721:00 +CPU= 27.888 Uploading trickle at timestep: 2592000 06:49:42 STEP 722 H= 722:00 +CPU= 28.164 06:50:23 STEP 723 H= 723:00 +CPU= 40.485 06:50:51 STEP 724 H= 724:00 +CPU= 28.097 06:51:20 STEP 725 H= 725:00 +CPU= 28.467 06:52:04 STEP 726 H= 726:00 +CPU= 44.022 06:52:30 STEP 727 H= 727:00 +CPU= 26.244 06:52:58 STEP 728 H= 728:00 +CPU= 27.954 06:53:40 STEP 729 H= 729:00 +CPU= 41.899 06:54:11 STEP 730 H= 730:00 +CPU= 30.885 06:54:40 STEP 731 H= 731:00 +CPU= 28.761 06:55:25 STEP 732 H= 732:00 +CPU= 45.575 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000732 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000732 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000732 06:55:53 STEP 733 H= 733:00 +CPU= 27.015 06:56:22 STEP 734 H= 734:00 +CPU= 28.986 06:57:04 STEP 735 H= 735:00 +CPU= 42.336 06:57:33 STEP 736 H= 736:00 +CPU= 28.476 06:58:01 STEP 737 H= 737:00 +CPU= 28.730 06:58:43 STEP 738 H= 738:00 +CPU= 41.391 06:59:10 STEP 739 H= 739:00 +CPU= 27.093 06:59:39 STEP 740 H= 740:00 +CPU= 28.751 07:00:22 STEP 741 H= 741:00 +CPU= 42.924 07:00:50 STEP 742 H= 742:00 +CPU= 28.302 07:01:19 STEP 743 H= 743:00 +CPU= 28.768 07:02:10 STEP 744 H= 744:00 +CPU= 51.088 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000744 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000744 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000720 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000720 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000720 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000732 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000732 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000732 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_11.zip Uploading the intermediate file: upload_file_11.zip 07:02:37 STEP 745 H= 745:00 +CPU= 26.951 Uploading trickle at timestep: 2678400 07:03:06 STEP 746 H= 746:00 +CPU= 28.349 07:03:48 STEP 747 H= 747:00 +CPU= 42.044 07:04:18 STEP 748 H= 748:00 +CPU= 29.790 07:04:46 STEP 749 H= 749:00 +CPU= 28.030 07:05:29 STEP 750 H= 750:00 +CPU= 43.747 07:05:57 STEP 751 H= 751:00 +CPU= 27.112 07:06:24 STEP 752 H= 752:00 +CPU= 26.928 07:07:06 STEP 753 H= 753:00 +CPU= 42.006 07:07:36 STEP 754 H= 754:00 +CPU= 30.266 07:08:04 STEP 755 H= 755:00 +CPU= 28.375 07:08:51 STEP 756 H= 756:00 +CPU= 46.625 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000756 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000756 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000756 07:09:20 STEP 757 H= 757:00 +CPU= 28.708 07:09:48 STEP 758 H= 758:00 +CPU= 28.467 07:10:31 STEP 759 H= 759:00 +CPU= 42.142 07:10:59 STEP 760 H= 760:00 +CPU= 28.349 07:11:27 STEP 761 H= 761:00 +CPU= 27.736 07:12:12 STEP 762 H= 762:00 +CPU= 45.194 07:12:39 STEP 763 H= 763:00 +CPU= 26.837 07:13:08 STEP 764 H= 764:00 +CPU= 28.618 07:13:50 STEP 765 H= 765:00 +CPU= 42.177 07:14:19 STEP 766 H= 766:00 +CPU= 29.336 07:14:48 STEP 767 H= 767:00 +CPU= 28.274 07:15:37 STEP 768 H= 768:00 +CPU= 48.880 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000768 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000768 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000744 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000756 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000756 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000756 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_12.zip Uploading the intermediate file: upload_file_12.zip Uploading trickle at timestep: 2764800 07:16:04 STEP 769 H= 769:00 +CPU= 27.500 07:16:33 STEP 770 H= 770:00 +CPU= 28.600 07:17:16 STEP 771 H= 771:00 +CPU= 43.304 07:17:46 STEP 772 H= 772:00 +CPU= 29.531 07:18:13 STEP 773 H= 773:00 +CPU= 27.196 07:18:58 STEP 774 H= 774:00 +CPU= 44.204 07:19:25 STEP 775 H= 775:00 +CPU= 26.864 07:19:54 STEP 776 H= 776:00 +CPU= 29.858 07:20:37 STEP 777 H= 777:00 +CPU= 42.599 07:21:05 STEP 778 H= 778:00 +CPU= 27.471 07:21:32 STEP 779 H= 779:00 +CPU= 26.997 07:22:19 STEP 780 H= 780:00 +CPU= 47.306 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000780 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000780 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000780 07:22:47 STEP 781 H= 781:00 +CPU= 27.848 07:23:15 STEP 782 H= 782:00 +CPU= 28.400 07:23:57 STEP 783 H= 783:00 +CPU= 41.692 07:24:25 STEP 784 H= 784:00 +CPU= 27.882 07:24:54 STEP 785 H= 785:00 +CPU= 28.709 07:25:37 STEP 786 H= 786:00 +CPU= 43.253 07:26:05 STEP 787 H= 787:00 +CPU= 27.439 07:26:33 STEP 788 H= 788:00 +CPU= 28.076 07:27:16 STEP 789 H= 789:00 +CPU= 43.089 07:27:45 STEP 790 H= 790:00 +CPU= 28.748 07:28:13 STEP 791 H= 791:00 +CPU= 28.099 07:29:04 STEP 792 H= 792:00 +CPU= 50.856 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000792 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000792 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000768 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000780 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000780 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_13.zip Uploading the intermediate file: upload_file_13.zip 07:29:31 STEP 793 H= 793:00 +CPU= 26.589 Uploading trickle at timestep: 2851200 07:30:01 STEP 794 H= 794:00 +CPU= 29.803 07:30:44 STEP 795 H= 795:00 +CPU= 43.415 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 29 MAX ETADOT VERTICAL VEL.= 4.003676801338262E-008 LON = 32.3437500000000 degrees LAT = -44.2987941306940 degrees 07:31:13 STEP 796 H= 796:00 +CPU= 29.153 07:31:39 STEP 797 H= 797:00 +CPU= 26.107 07:32:24 STEP 798 H= 798:00 +CPU= 44.228 07:32:52 STEP 799 H= 799:00 +CPU= 27.847 07:33:20 STEP 800 H= 800:00 +CPU= 27.912 07:34:01 STEP 801 H= 801:00 +CPU= 41.612 07:34:29 STEP 802 H= 802:00 +CPU= 27.660 07:34:57 STEP 803 H= 803:00 +CPU= 27.488 07:35:44 STEP 804 H= 804:00 +CPU= 47.655 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000804 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000804 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000804 07:36:13 STEP 805 H= 805:00 +CPU= 28.557 07:36:41 STEP 806 H= 806:00 +CPU= 27.725 07:37:24 STEP 807 H= 807:00 +CPU= 42.905 07:37:53 STEP 808 H= 808:00 +CPU= 29.324 07:38:21 STEP 809 H= 809:00 +CPU= 27.641 07:39:04 STEP 810 H= 810:00 +CPU= 43.173 07:39:32 STEP 811 H= 811:00 +CPU= 27.670 07:39:59 STEP 812 H= 812:00 +CPU= 27.166 07:40:42 STEP 813 H= 813:00 +CPU= 42.808 07:41:10 STEP 814 H= 814:00 +CPU= 28.005 07:41:38 STEP 815 H= 815:00 +CPU= 27.747 07:42:28 STEP 816 H= 816:00 +CPU= 50.240 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000816 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000816 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000792 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000804 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000804 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_14.zip Uploading the intermediate file: upload_file_14.zip Uploading trickle at timestep: 2937600 07:42:57 STEP 817 H= 817:00 +CPU= 28.360 07:43:26 STEP 818 H= 818:00 +CPU= 28.928 07:44:09 STEP 819 H= 819:00 +CPU= 43.390 07:44:37 STEP 820 H= 820:00 +CPU= 28.093 07:45:04 STEP 821 H= 821:00 +CPU= 26.635 07:45:48 STEP 822 H= 822:00 +CPU= 43.849 07:46:15 STEP 823 H= 823:00 +CPU= 27.394 07:46:43 STEP 824 H= 824:00 +CPU= 27.685 07:47:25 STEP 825 H= 825:00 +CPU= 41.612 07:47:53 STEP 826 H= 826:00 +CPU= 28.200 07:48:21 STEP 827 H= 827:00 +CPU= 28.390 07:49:09 STEP 828 H= 828:00 +CPU= 47.203 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000828 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000828 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000828 07:49:38 STEP 829 H= 829:00 +CPU= 28.775 07:50:06 STEP 830 H= 830:00 +CPU= 28.103 07:50:49 STEP 831 H= 831:00 +CPU= 43.171 07:51:18 STEP 832 H= 832:00 +CPU= 29.093 07:51:45 STEP 833 H= 833:00 +CPU= 26.700 07:52:26 STEP 834 H= 834:00 +CPU= 41.505 07:52:55 STEP 835 H= 835:00 +CPU= 28.956 07:53:25 STEP 836 H= 836:00 +CPU= 29.093 07:54:06 STEP 837 H= 837:00 +CPU= 41.591 07:54:35 STEP 838 H= 838:00 +CPU= 28.311 07:55:04 STEP 839 H= 839:00 +CPU= 28.926 07:55:54 STEP 840 H= 840:00 +CPU= 50.271 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000840 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000840 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000816 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000828 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000828 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_15.zip Uploading the intermediate file: upload_file_15.zip Uploading trickle at timestep: 3024000 07:56:23 STEP 841 H= 841:00 +CPU= 29.420 07:56:52 STEP 842 H= 842:00 +CPU= 27.980 07:57:34 STEP 843 H= 843:00 +CPU= 41.953 07:58:03 STEP 844 H= 844:00 +CPU= 29.713 07:58:32 STEP 845 H= 845:00 +CPU= 28.946 07:59:16 STEP 846 H= 846:00 +CPU= 43.460 07:59:44 STEP 847 H= 847:00 +CPU= 28.312 08:00:13 STEP 848 H= 848:00 +CPU= 28.718 08:00:56 STEP 849 H= 849:00 +CPU= 42.752 08:01:25 STEP 850 H= 850:00 +CPU= 28.872 08:01:51 STEP 851 H= 851:00 +CPU= 26.677 08:02:38 STEP 852 H= 852:00 +CPU= 46.754 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000852 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000852 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000852 08:03:07 STEP 853 H= 853:00 +CPU= 28.278 08:03:37 STEP 854 H= 854:00 +CPU= 30.138 08:04:19 STEP 855 H= 855:00 +CPU= 42.023 08:04:49 STEP 856 H= 856:00 +CPU= 29.502 08:05:17 STEP 857 H= 857:00 +CPU= 28.721 08:06:00 STEP 858 H= 858:00 +CPU= 42.912 08:06:30 STEP 859 H= 859:00 +CPU= 29.453 08:06:57 STEP 860 H= 860:00 +CPU= 27.172 08:07:40 STEP 861 H= 861:00 +CPU= 42.375 08:08:08 STEP 862 H= 862:00 +CPU= 28.439 08:08:38 STEP 863 H= 863:00 +CPU= 29.970 08:09:28 STEP 864 H= 864:00 +CPU= 50.040 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000864 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000864 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000840 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000852 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000852 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000852 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_16.zip Uploading the intermediate file: upload_file_16.zip Uploading trickle at timestep: 3110400 08:09:56 STEP 865 H= 865:00 +CPU= 27.213 08:10:24 STEP 866 H= 866:00 +CPU= 28.263 08:11:06 STEP 867 H= 867:00 +CPU= 42.266 08:11:36 STEP 868 H= 868:00 +CPU= 29.326 08:12:03 STEP 869 H= 869:00 +CPU= 27.070 08:12:46 STEP 870 H= 870:00 +CPU= 43.318 08:13:14 STEP 871 H= 871:00 +CPU= 27.767 08:13:44 STEP 872 H= 872:00 +CPU= 29.598 08:14:26 STEP 873 H= 873:00 +CPU= 42.370 08:14:55 STEP 874 H= 874:00 +CPU= 28.714 08:15:24 STEP 875 H= 875:00 +CPU= 29.462 08:16:11 STEP 876 H= 876:00 +CPU= 46.796 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000876 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000876 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000876 08:16:39 STEP 877 H= 877:00 +CPU= 28.143 08:17:06 STEP 878 H= 878:00 +CPU= 26.888 08:17:47 STEP 879 H= 879:00 +CPU= 40.617 08:18:15 STEP 880 H= 880:00 +CPU= 27.810 08:18:43 STEP 881 H= 881:00 +CPU= 28.265 08:19:28 STEP 882 H= 882:00 +CPU= 44.980 08:19:56 STEP 883 H= 883:00 +CPU= 27.458 08:20:25 STEP 884 H= 884:00 +CPU= 29.065 08:21:08 STEP 885 H= 885:00 +CPU= 43.099 08:21:38 STEP 886 H= 886:00 +CPU= 29.986 08:22:07 STEP 887 H= 887:00 +CPU= 28.751 08:22:56 STEP 888 H= 888:00 +CPU= 48.872 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000888 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000888 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000864 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000876 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000876 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_17.zip Uploading the intermediate file: upload_file_17.zip 08:23:23 STEP 889 H= 889:00 +CPU= 27.216 Uploading trickle at timestep: 3196800 08:23:54 STEP 890 H= 890:00 +CPU= 30.482 08:24:38 STEP 891 H= 891:00 +CPU= 44.137 08:25:07 STEP 892 H= 892:00 +CPU= 28.513 08:25:35 STEP 893 H= 893:00 +CPU= 28.677 08:26:18 STEP 894 H= 894:00 +CPU= 42.538 08:26:46 STEP 895 H= 895:00 +CPU= 27.789 08:27:15 STEP 896 H= 896:00 +CPU= 29.614 08:27:57 STEP 897 H= 897:00 +CPU= 41.895 08:28:25 STEP 898 H= 898:00 +CPU= 27.904 08:28:53 STEP 899 H= 899:00 +CPU= 27.903 08:29:41 STEP 900 H= 900:00 +CPU= 47.180 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000900 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000900 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000900 08:30:08 STEP 901 H= 901:00 +CPU= 27.066 08:30:37 STEP 902 H= 902:00 +CPU= 29.485 08:31:20 STEP 903 H= 903:00 +CPU= 42.402 08:31:48 STEP 904 H= 904:00 +CPU= 27.712 08:32:17 STEP 905 H= 905:00 +CPU= 29.391 08:33:00 STEP 906 H= 906:00 +CPU= 42.456 08:33:27 STEP 907 H= 907:00 +CPU= 27.052 08:33:56 STEP 908 H= 908:00 +CPU= 28.930 08:34:39 STEP 909 H= 909:00 +CPU= 43.690 08:35:08 STEP 910 H= 910:00 +CPU= 28.163 08:35:38 STEP 911 H= 911:00 +CPU= 30.127 08:36:28 STEP 912 H= 912:00 +CPU= 50.094 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000912 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000912 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000912 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000888 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMGGhpi1+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMSHhpi1+000900 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12163774/ICMUAhpi1+000900 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0685_2021050100_123_945_12163774_0_r1232042209_18.zip Uploading the intermediate file: upload_file_18.zip Uploading trickle at timestep: 3283200 08:36:57 STEP 913 H= 913:00 +CPU= 28.379 08:37:27 STEP 914 H= 914:00 +CPU= 30.531 08:38:07 STEP 915 H= 915:00 +CPU= 39.363 08:38:34 STEP 916 H= 916:00 +CPU= 27.491 08:39:02 STEP 917 H= 917:00 +CPU= 27.443 08:39:45 STEP 918 H= 918:00 +CPU= 43.789 08:40:12 STEP 919 H= 919:00 +CPU= 26.561 08:40:42 STEP 920 H= 920:00 +CPU= 29.707 08:41:24 STEP 921 H= 921:00 +CPU= 42.375 08:41:53 STEP 922 H= 922:00 +CPU= 28.390 08:42:23 STEP 923 H= 923:00 +CPU= 29.738 08:43:08 STEP 924 H= 924:00 +CPU= 44.986 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGhpi1+000924 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHhpi1+000924 Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAhpi1+000924 08:43:35 STEP 925 H= 925:00 +CPU= 26.845 08:44:03 STEP 926 H= 926:00 +CPU= 28.388 08:44:47 STEP 927 H= 927:00 +CPU= 43.683 08:45:15 STEP 928 H= 928:00 +CPU= 27.899 08:45:43 STEP 929 H= 929:00 +CPU= 27.826 08:46:26 STEP 930 H= 930:00 +CPU= 43.117 08:46:53 STEP 931 H= 931:00 +CPU= 27.107 08:47:17 STEP 932 H= 932:00 +CPU= 23.960 08:47:53 STEP 933 H= 933:00 +CPU= 35.463 08:48:15 STEP 934 H= 934:00 +CPU= 22.442 08:48:40 STEP 935 H= 935:00 +CPU= 24.524 The child process terminated with status: 0 ..Failed, model did not complete successfully </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 15:41:35 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 3,283,200 | 14,570 | 0.0044 |
09 May 2023 15:41:35 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 3,196,800 | 13,759 | 0.0043 |
09 May 2023 15:41:35 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 3,110,400 | 12,948 | 0.0042 |
09 May 2023 15:41:35 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 3,024,000 | 12,137 | 0.0040 |
09 May 2023 15:41:32 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,937,600 | 11,336 | 0.0039 |
09 May 2023 15:41:32 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,851,200 | 10,535 | 0.0037 |
09 May 2023 15:41:32 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,764,800 | 9,724 | 0.0035 |
09 May 2023 15:41:32 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,678,400 | 8,923 | 0.0033 |
09 May 2023 15:41:29 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,592,000 | 8,122 | 0.0031 |
09 May 2023 15:41:29 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,505,600 | 7,311 | 0.0029 |
09 May 2023 15:41:29 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,419,200 | 6,510 | 0.0027 |
09 May 2023 15:41:29 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,332,800 | 5,699 | 0.0024 |
09 May 2023 15:41:29 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,246,400 | 4,898 | 0.0022 |
09 May 2023 15:41:25 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,160,000 | 4,087 | 0.0019 |
09 May 2023 15:41:25 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 2,073,600 | 3,286 | 0.0016 |
09 May 2023 15:41:25 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 1,987,200 | 2,475 | 0.0012 |
09 May 2023 15:41:25 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 1,900,800 | 1,664 | 0.0009 |
09 May 2023 15:41:22 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 1,814,400 | 853 | 0.0005 |
09 May 2023 15:41:22 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 1,728,000 | 16,271 | 0.0094 |
09 May 2023 15:41:22 | 1522983 | 22245718 | oifs_43r3_ps_0685_2021050100_123_945_12163774_0 | 1,641,600 | 15,460 | 0.0094 |
©2024 cpdn.org