Name | oifs_43r3_00wj_2019110100_123_993_12214609_0 |
Workunit | 12214609 |
Created | 24 Feb 2023, 15:37:10 UTC |
Sent | 24 Feb 2023, 15:45:17 UTC |
Report deadline | 25 Apr 2023, 15:45:17 UTC |
Received | 24 Feb 2023, 22:50:06 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 5 (0x00000005) Unknown error code |
Computer ID | 1539146 |
Run time | 6 hours 10 min 16 sec |
CPU time | 6 hours 10 min 3 sec |
Validate state | Invalid |
Credit | 4,838.40 |
Device peak FLOPS | 8.36 GFLOPS |
Application version | OpenIFS 43r3 v1.21 x86_64-pc-linux-gnu |
Peak working set size | 4,794.07 MB |
Peak swap size | 4,974.27 MB |
Peak disk usage | 1,264.98 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> process exited with code 5 (0x5, -251)</message> <stderr_txt> 8.317 10:57:37 STEP 365 H= 365:00 +CPU= 11.755 10:58:27 STEP 366 H= 366:00 +CPU= 49.382 10:58:46 STEP 367 H= 367:00 +CPU= 19.643 10:59:16 STEP 368 H= 368:00 +CPU= 29.324 11:00:02 STEP 369 H= 369:00 +CPU= 46.738 11:00:37 STEP 370 H= 370:00 +CPU= 34.830 11:01:07 STEP 371 H= 371:00 +CPU= 29.325 11:01:48 STEP 372 H= 372:00 +CPU= 41.199 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000372 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000372 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000372 11:01:58 STEP 373 H= 373:00 +CPU= 10.687 11:02:09 STEP 374 H= 374:00 +CPU= 10.618 11:02:25 STEP 375 H= 375:00 +CPU= 15.570 11:02:34 STEP 376 H= 376:00 +CPU= 9.270 11:02:43 STEP 377 H= 377:00 +CPU= 8.841 11:02:57 STEP 378 H= 378:00 +CPU= 14.065 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 7 MAX ETADOT VERTICAL VEL.= 1.136750659685818E-007 LON = 143.750000000000 degrees LAT = 39.8128523737712 degrees 11:03:05 STEP 379 H= 379:00 +CPU= 8.332 11:03:14 STEP 380 H= 380:00 +CPU= 8.372 11:03:27 STEP 381 H= 381:00 +CPU= 13.063 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 27 MAX ETADOT VERTICAL VEL.= 2.730222038111935E-007 LON = 144.843750000000 degrees LAT = 42.0558239509352 degrees 11:03:35 STEP 382 H= 382:00 +CPU= 8.142 11:03:46 STEP 383 H= 383:00 +CPU= 11.614 11:04:38 STEP 384 H= 384:00 +CPU= 51.896 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000384 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000384 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000384 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000360 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000360 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000360 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000372 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000372 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000372 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_15.zip Uploading the intermediate file: upload_file_15.zip 11:05:08 STEP 385 H= 385:00 +CPU= 29.477 11:05:34 STEP 386 H= 386:00 +CPU= 26.432 SMILAG TRAJECTORY UNDERGROUND 1 TIMES. POINT= 28 MAX ETADOT VERTICAL VEL.= 2.098991060588269E-007 LON = 146.250000000000 degrees LAT = 45.4202786265483 degrees 11:06:25 STEP 387 H= 387:00 +CPU= 50.348 11:06:53 STEP 388 H= 388:00 +CPU= 28.078 11:07:26 STEP 389 H= 389:00 +CPU= 33.584 11:08:15 STEP 390 H= 390:00 +CPU= 48.854 11:08:26 STEP 391 H= 391:00 +CPU= 10.684 11:08:36 STEP 392 H= 392:00 +CPU= 10.189 11:08:51 STEP 393 H= 393:00 +CPU= 14.854 11:09:00 STEP 394 H= 394:00 +CPU= 8.857 11:09:08 STEP 395 H= 395:00 +CPU= 8.499 11:09:23 STEP 396 H= 396:00 +CPU= 14.719 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000396 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000396 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000396 11:09:31 STEP 397 H= 397:00 +CPU= 7.988 11:09:39 STEP 398 H= 398:00 +CPU= 8.202 11:09:52 STEP 399 H= 399:00 +CPU= 12.942 11:10:19 STEP 400 H= 400:00 +CPU= 26.800 11:10:48 STEP 401 H= 401:00 +CPU= 28.848 11:11:36 STEP 402 H= 402:00 +CPU= 48.555 11:11:59 STEP 403 H= 403:00 +CPU= 22.826 11:12:32 STEP 404 H= 404:00 +CPU= 32.576 11:13:15 STEP 405 H= 405:00 +CPU= 43.269 11:13:46 STEP 406 H= 406:00 +CPU= 30.401 11:14:07 STEP 407 H= 407:00 +CPU= 21.302 11:14:29 STEP 408 H= 408:00 +CPU= 21.706 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000408 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000408 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000408 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000384 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000384 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000384 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000396 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000396 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000396 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_16.zip Uploading the intermediate file: upload_file_16.zip 11:14:39 STEP 409 H= 409:00 +CPU= 10.315 11:14:49 STEP 410 H= 410:00 +CPU= 10.157 11:15:04 STEP 411 H= 411:00 +CPU= 14.694 11:15:12 STEP 412 H= 412:00 +CPU= 8.759 11:15:21 STEP 413 H= 413:00 +CPU= 8.171 11:15:34 STEP 414 H= 414:00 +CPU= 13.190 11:15:42 STEP 415 H= 415:00 +CPU= 7.817 11:15:52 STEP 416 H= 416:00 +CPU= 10.013 11:16:34 STEP 417 H= 417:00 +CPU= 42.808 11:16:55 STEP 418 H= 418:00 +CPU= 20.940 11:17:20 STEP 419 H= 419:00 +CPU= 24.099 11:18:16 STEP 420 H= 420:00 +CPU= 56.596 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000420 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000420 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000420 11:18:48 STEP 421 H= 421:00 +CPU= 32.107 11:19:20 STEP 422 H= 422:00 +CPU= 32.031 11:20:08 STEP 423 H= 423:00 +CPU= 47.540 11:20:19 STEP 424 H= 424:00 +CPU= 11.392 11:20:30 STEP 425 H= 425:00 +CPU= 10.981 11:20:47 STEP 426 H= 426:00 +CPU= 16.543 11:20:56 STEP 427 H= 427:00 +CPU= 9.315 11:21:05 STEP 428 H= 428:00 +CPU= 9.196 11:21:19 STEP 429 H= 429:00 +CPU= 13.795 11:21:27 STEP 430 H= 430:00 +CPU= 8.297 11:21:36 STEP 431 H= 431:00 +CPU= 8.199 11:21:50 STEP 432 H= 432:00 +CPU= 14.785 11:21:58 STEP 433 H= 433:00 +CPU= 7.804 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000432 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000432 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000432 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000408 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000408 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000408 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000420 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000420 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000420 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_17.zip Uploading the intermediate file: upload_file_17.zip 11:22:06 STEP 434 H= 434:00 +CPU= 8.064 11:22:38 STEP 435 H= 435:00 +CPU= 31.933 11:23:00 STEP 436 H= 436:00 +CPU= 22.134 11:23:27 STEP 437 H= 437:00 +CPU= 27.091 11:23:59 STEP 438 H= 438:00 +CPU= 31.585 11:24:26 STEP 439 H= 439:00 +CPU= 26.757 11:24:55 STEP 440 H= 440:00 +CPU= 29.269 11:25:36 STEP 441 H= 441:00 +CPU= 40.645 11:26:08 STEP 442 H= 442:00 +CPU= 32.075 11:26:26 STEP 443 H= 443:00 +CPU= 17.794 11:26:47 STEP 444 H= 444:00 +CPU= 20.934 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000444 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000444 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000444 11:26:57 STEP 445 H= 445:00 +CPU= 10.717 11:27:08 STEP 446 H= 446:00 +CPU= 10.304 11:27:22 STEP 447 H= 447:00 +CPU= 14.508 11:27:31 STEP 448 H= 448:00 +CPU= 8.696 11:27:39 STEP 449 H= 449:00 +CPU= 8.335 11:27:53 STEP 450 H= 450:00 +CPU= 13.552 11:28:01 STEP 451 H= 451:00 +CPU= 7.955 11:28:09 STEP 452 H= 452:00 +CPU= 8.151 11:28:22 STEP 453 H= 453:00 +CPU= 12.840 11:28:30 STEP 454 H= 454:00 +CPU= 8.035 11:28:43 STEP 455 H= 455:00 +CPU= 12.775 11:29:30 STEP 456 H= 456:00 +CPU= 47.438 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000456 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000456 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000456 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000432 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000432 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000432 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000444 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000444 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000444 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_18.zip Uploading the intermediate file: upload_file_18.zip 11:29:56 STEP 457 H= 457:00 +CPU= 26.136 11:30:26 STEP 458 H= 458:00 +CPU= 29.420 11:31:11 STEP 459 H= 459:00 +CPU= 45.624 11:31:44 STEP 460 H= 460:00 +CPU= 33.194 11:32:17 STEP 461 H= 461:00 +CPU= 32.104 11:33:02 STEP 462 H= 462:00 +CPU= 45.611 11:33:13 STEP 463 H= 463:00 +CPU= 11.012 11:33:24 STEP 464 H= 464:00 +CPU= 10.738 11:33:40 STEP 465 H= 465:00 +CPU= 15.634 11:33:49 STEP 466 H= 466:00 +CPU= 9.188 11:33:57 STEP 467 H= 467:00 +CPU= 8.578 11:34:12 STEP 468 H= 468:00 +CPU= 14.824 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000468 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000468 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000468 11:34:20 STEP 469 H= 469:00 +CPU= 7.863 11:34:28 STEP 470 H= 470:00 +CPU= 8.074 11:34:41 STEP 471 H= 471:00 +CPU= 12.781 11:34:53 STEP 472 H= 472:00 +CPU= 12.012 11:35:21 STEP 473 H= 473:00 +CPU= 28.244 11:36:00 STEP 474 H= 474:00 +CPU= 38.718 11:36:30 STEP 475 H= 475:00 +CPU= 29.884 11:36:59 STEP 476 H= 476:00 +CPU= 29.063 11:37:46 STEP 477 H= 477:00 +CPU= 46.684 11:38:16 STEP 478 H= 478:00 +CPU= 30.545 11:38:50 STEP 479 H= 479:00 +CPU= 33.742 11:39:17 STEP 480 H= 480:00 +CPU= 27.067 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000480 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000480 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000480 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000456 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000456 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000456 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000468 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000468 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000468 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_19.zip Uploading the intermediate file: upload_file_19.zip 11:39:28 STEP 481 H= 481:00 +CPU= 10.846 11:39:39 STEP 482 H= 482:00 +CPU= 10.664 Uploading trickle at timestep: 1728000 11:39:54 STEP 483 H= 483:00 +CPU= 15.030 11:40:03 STEP 484 H= 484:00 +CPU= 8.977 11:40:11 STEP 485 H= 485:00 +CPU= 8.472 11:40:25 STEP 486 H= 486:00 +CPU= 13.610 11:40:33 STEP 487 H= 487:00 +CPU= 7.944 11:40:41 STEP 488 H= 488:00 +CPU= 8.144 11:41:06 STEP 489 H= 489:00 +CPU= 25.342 11:41:25 STEP 490 H= 490:00 +CPU= 19.107 11:41:52 STEP 491 H= 491:00 +CPU= 26.416 11:42:47 STEP 492 H= 492:00 +CPU= 55.703 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000492 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000492 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000492 11:43:17 STEP 493 H= 493:00 +CPU= 29.277 11:43:49 STEP 494 H= 494:00 +CPU= 32.166 11:44:28 STEP 495 H= 495:00 +CPU= 38.977 11:44:49 STEP 496 H= 496:00 +CPU= 20.716 11:45:18 STEP 497 H= 497:00 +CPU= 29.265 11:45:35 STEP 498 H= 498:00 +CPU= 17.535 11:45:46 STEP 499 H= 499:00 +CPU= 10.230 11:45:56 STEP 500 H= 500:00 +CPU= 9.889 11:46:10 STEP 501 H= 501:00 +CPU= 14.363 11:46:19 STEP 502 H= 502:00 +CPU= 8.682 11:46:27 STEP 503 H= 503:00 +CPU= 8.364 11:46:42 STEP 504 H= 504:00 +CPU= 14.809 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000504 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000504 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000504 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000480 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000480 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000480 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000492 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000492 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000492 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_20.zip Uploading the intermediate file: upload_file_20.zip 11:46:50 STEP 505 H= 505:00 +CPU= 7.889 11:46:58 STEP 506 H= 506:00 +CPU= 8.008 11:47:33 STEP 507 H= 507:00 +CPU= 35.248 11:48:02 STEP 508 H= 508:00 +CPU= 29.375 11:48:30 STEP 509 H= 509:00 +CPU= 27.928 11:49:18 STEP 510 H= 510:00 +CPU= 47.367 11:49:52 STEP 511 H= 511:00 +CPU= 34.135 11:50:26 STEP 512 H= 512:00 +CPU= 33.875 11:51:10 STEP 513 H= 513:00 +CPU= 44.635 11:51:35 STEP 514 H= 514:00 +CPU= 24.748 11:51:46 STEP 515 H= 515:00 +CPU= 10.623 11:52:03 STEP 516 H= 516:00 +CPU= 17.165 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000516 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000516 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000516 11:52:12 STEP 517 H= 517:00 +CPU= 8.999 11:52:21 STEP 518 H= 518:00 +CPU= 8.948 11:52:35 STEP 519 H= 519:00 +CPU= 13.795 11:52:43 STEP 520 H= 520:00 +CPU= 8.440 11:52:51 STEP 521 H= 521:00 +CPU= 8.304 11:53:31 STEP 522 H= 522:00 +CPU= 39.702 11:53:55 STEP 523 H= 523:00 +CPU= 24.128 11:54:18 STEP 524 H= 524:00 +CPU= 22.607 11:55:05 STEP 525 H= 525:00 +CPU= 46.623 11:55:32 STEP 526 H= 526:00 +CPU= 26.956 11:56:00 STEP 527 H= 527:00 +CPU= 28.719 11:56:57 STEP 528 H= 528:00 +CPU= 56.781 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000528 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000528 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000528 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000504 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000504 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000504 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000516 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000516 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000516 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_21.zip Uploading the intermediate file: upload_file_21.zip 11:57:14 STEP 529 H= 529:00 +CPU= 16.665 11:57:25 STEP 530 H= 530:00 +CPU= 11.420 11:57:41 STEP 531 H= 531:00 +CPU= 16.212 11:57:51 STEP 532 H= 532:00 +CPU= 9.802 11:58:00 STEP 533 H= 533:00 +CPU= 9.228 11:58:15 STEP 534 H= 534:00 +CPU= 14.411 11:58:23 STEP 535 H= 535:00 +CPU= 8.301 11:58:32 STEP 536 H= 536:00 +CPU= 8.497 11:58:45 STEP 537 H= 537:00 +CPU= 13.080 11:59:02 STEP 538 H= 538:00 +CPU= 17.737 11:59:34 STEP 539 H= 539:00 +CPU= 31.110 12:00:30 STEP 540 H= 540:00 +CPU= 56.203 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000540 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000540 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000540 12:01:00 STEP 541 H= 541:00 +CPU= 30.652 12:01:23 STEP 542 H= 542:00 +CPU= 22.562 12:02:11 STEP 543 H= 543:00 +CPU= 47.626 12:02:44 STEP 544 H= 544:00 +CPU= 32.858 12:03:03 STEP 545 H= 545:00 +CPU= 19.920 12:03:22 STEP 546 H= 546:00 +CPU= 18.348 12:03:33 STEP 547 H= 547:00 +CPU= 10.812 12:03:43 STEP 548 H= 548:00 +CPU= 10.767 12:03:59 STEP 549 H= 549:00 +CPU= 15.627 12:04:08 STEP 550 H= 550:00 +CPU= 8.943 12:04:17 STEP 551 H= 551:00 +CPU= 8.490 12:04:32 STEP 552 H= 552:00 +CPU= 15.440 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000552 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000552 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000552 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000528 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000528 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000528 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000540 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000540 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000540 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_22.zip Uploading the intermediate file: upload_file_22.zip 12:04:40 STEP 553 H= 553:00 +CPU= 8.068 12:04:48 STEP 554 H= 554:00 +CPU= 8.127 12:05:02 STEP 555 H= 555:00 +CPU= 13.709 12:05:29 STEP 556 H= 556:00 +CPU= 27.554 12:06:00 STEP 557 H= 557:00 +CPU= 30.431 12:06:40 STEP 558 H= 558:00 +CPU= 39.779 12:07:11 STEP 559 H= 559:00 +CPU= 31.597 12:07:45 STEP 560 H= 560:00 +CPU= 33.928 12:08:35 STEP 561 H= 561:00 +CPU= 49.952 12:09:08 STEP 562 H= 562:00 +CPU= 32.724 12:09:27 STEP 563 H= 563:00 +CPU= 18.747 12:09:47 STEP 564 H= 564:00 +CPU= 19.769 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000564 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000564 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000564 12:09:56 STEP 565 H= 565:00 +CPU= 9.768 12:10:06 STEP 566 H= 566:00 +CPU= 9.403 12:10:20 STEP 567 H= 567:00 +CPU= 14.248 12:10:29 STEP 568 H= 568:00 +CPU= 8.591 12:10:37 STEP 569 H= 569:00 +CPU= 8.179 12:10:50 STEP 570 H= 570:00 +CPU= 13.309 12:10:58 STEP 571 H= 571:00 +CPU= 7.836 12:11:06 STEP 572 H= 572:00 +CPU= 8.059 12:11:45 STEP 573 H= 573:00 +CPU= 39.372 12:12:18 STEP 574 H= 574:00 +CPU= 32.154 12:12:49 STEP 575 H= 575:00 +CPU= 31.642 12:13:49 STEP 576 H= 576:00 +CPU= 59.933 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000576 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000576 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000576 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000552 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000552 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000552 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000564 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000564 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000564 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_23.zip Uploading the intermediate file: upload_file_23.zip 12:14:13 STEP 577 H= 577:00 +CPU= 23.658 12:14:38 STEP 578 H= 578:00 +CPU= 25.173 12:15:20 STEP 579 H= 579:00 +CPU= 42.099 12:15:33 STEP 580 H= 580:00 +CPU= 12.630 12:15:44 STEP 581 H= 581:00 +CPU= 10.943 12:16:00 STEP 582 H= 582:00 +CPU= 16.116 12:16:09 STEP 583 H= 583:00 +CPU= 9.345 12:16:18 STEP 584 H= 584:00 +CPU= 9.178 12:16:32 STEP 585 H= 585:00 +CPU= 14.059 12:16:41 STEP 586 H= 586:00 +CPU= 8.757 12:16:49 STEP 587 H= 587:00 +CPU= 8.255 12:17:04 STEP 588 H= 588:00 +CPU= 14.190 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000588 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000588 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000588 12:17:12 STEP 589 H= 589:00 +CPU= 7.958 12:17:20 STEP 590 H= 590:00 +CPU= 8.040 12:17:43 STEP 591 H= 591:00 +CPU= 23.276 12:18:16 STEP 592 H= 592:00 +CPU= 33.236 12:18:45 STEP 593 H= 593:00 +CPU= 28.445 12:19:32 STEP 594 H= 594:00 +CPU= 47.745 12:19:57 STEP 595 H= 595:00 +CPU= 24.395 12:20:28 STEP 596 H= 596:00 +CPU= 31.032 12:21:13 STEP 597 H= 597:00 +CPU= 45.396 12:21:43 STEP 598 H= 598:00 +CPU= 30.005 12:22:01 STEP 599 H= 599:00 +CPU= 18.144 12:22:21 STEP 600 H= 600:00 +CPU= 19.435 12:22:30 STEP 601 H= 601:00 +CPU= 9.232 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000600 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000600 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000600 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000576 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000576 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000576 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000588 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000588 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000588 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_24.zip Uploading the intermediate file: upload_file_24.zip 12:22:39 STEP 602 H= 602:00 +CPU= 8.878 12:22:53 STEP 603 H= 603:00 +CPU= 13.748 12:23:01 STEP 604 H= 604:00 +CPU= 8.427 12:23:09 STEP 605 H= 605:00 +CPU= 8.010 12:23:31 STEP 606 H= 606:00 +CPU= 21.343 12:24:03 STEP 607 H= 607:00 +CPU= 32.245 12:24:36 STEP 608 H= 608:00 +CPU= 32.931 12:25:16 STEP 609 H= 609:00 +CPU= 40.665 12:25:47 STEP 610 H= 610:00 +CPU= 30.047 12:26:10 STEP 611 H= 611:00 +CPU= 23.568 12:26:54 STEP 612 H= 612:00 +CPU= 43.555 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000612 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000612 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000612 12:27:25 STEP 613 H= 613:00 +CPU= 31.436 12:27:48 STEP 614 H= 614:00 +CPU= 22.419 12:28:03 STEP 615 H= 615:00 +CPU= 15.881 12:28:13 STEP 616 H= 616:00 +CPU= 9.418 12:28:22 STEP 617 H= 617:00 +CPU= 8.945 12:28:36 STEP 618 H= 618:00 +CPU= 14.237 12:28:44 STEP 619 H= 619:00 +CPU= 8.363 12:28:53 STEP 620 H= 620:00 +CPU= 8.268 12:29:06 STEP 621 H= 621:00 +CPU= 12.836 12:29:14 STEP 622 H= 622:00 +CPU= 8.051 12:29:23 STEP 623 H= 623:00 +CPU= 9.394 12:30:15 STEP 624 H= 624:00 +CPU= 52.081 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000624 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000624 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000624 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000600 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000600 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000600 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000612 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000612 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000612 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_25.zip Uploading the intermediate file: upload_file_25.zip 12:30:40 STEP 625 H= 625:00 +CPU= 25.014 12:31:05 STEP 626 H= 626:00 +CPU= 24.763 12:31:51 STEP 627 H= 627:00 +CPU= 45.908 12:32:22 STEP 628 H= 628:00 +CPU= 31.537 12:32:53 STEP 629 H= 629:00 +CPU= 30.386 12:33:34 STEP 630 H= 630:00 +CPU= 40.934 12:33:46 STEP 631 H= 631:00 +CPU= 11.766 12:33:57 STEP 632 H= 632:00 +CPU= 11.746 12:34:14 STEP 633 H= 633:00 +CPU= 16.418 12:34:24 STEP 634 H= 634:00 +CPU= 9.775 12:34:32 STEP 635 H= 635:00 +CPU= 8.902 12:34:48 STEP 636 H= 636:00 +CPU= 15.191 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000636 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000636 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000636 12:34:56 STEP 637 H= 637:00 +CPU= 8.253 12:35:04 STEP 638 H= 638:00 +CPU= 8.414 12:35:17 STEP 639 H= 639:00 +CPU= 12.899 12:35:25 STEP 640 H= 640:00 +CPU= 8.048 12:35:49 STEP 641 H= 641:00 +CPU= 23.485 12:36:26 STEP 642 H= 642:00 +CPU= 37.556 12:36:56 STEP 643 H= 643:00 +CPU= 29.817 12:37:21 STEP 644 H= 644:00 +CPU= 24.754 12:38:06 STEP 645 H= 645:00 +CPU= 44.655 12:38:31 STEP 646 H= 646:00 +CPU= 25.328 12:38:51 STEP 647 H= 647:00 +CPU= 20.034 12:39:31 STEP 648 H= 648:00 +CPU= 40.155 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000648 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000648 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000648 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000624 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000624 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000624 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000636 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000636 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000636 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_26.zip Uploading the intermediate file: upload_file_26.zip 12:39:43 STEP 649 H= 649:00 +CPU= 12.033 12:39:55 STEP 650 H= 650:00 +CPU= 11.757 12:40:13 STEP 651 H= 651:00 +CPU= 17.985 12:40:24 STEP 652 H= 652:00 +CPU= 11.227 12:40:34 STEP 653 H= 653:00 +CPU= 10.326 12:40:49 STEP 654 H= 654:00 +CPU= 14.665 12:40:58 STEP 655 H= 655:00 +CPU= 8.352 12:41:06 STEP 656 H= 656:00 +CPU= 8.411 12:41:19 STEP 657 H= 657:00 +CPU= 13.096 12:41:41 STEP 658 H= 658:00 +CPU= 22.407 12:42:08 STEP 659 H= 659:00 +CPU= 26.665 12:42:56 STEP 660 H= 660:00 +CPU= 47.948 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000660 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000660 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000660 12:43:26 STEP 661 H= 661:00 +CPU= 29.496 12:43:56 STEP 662 H= 662:00 +CPU= 30.357 12:44:31 STEP 663 H= 663:00 +CPU= 35.088 12:44:59 STEP 664 H= 664:00 +CPU= 27.569 12:45:29 STEP 665 H= 665:00 +CPU= 30.199 12:45:48 STEP 666 H= 666:00 +CPU= 19.244 12:45:59 STEP 667 H= 667:00 +CPU= 10.475 12:46:09 STEP 668 H= 668:00 +CPU= 10.732 12:46:25 STEP 669 H= 669:00 +CPU= 15.812 12:46:35 STEP 670 H= 670:00 +CPU= 9.610 12:46:44 STEP 671 H= 671:00 +CPU= 9.265 12:47:00 STEP 672 H= 672:00 +CPU= 15.972 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000672 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000672 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000672 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000648 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000648 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000648 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000660 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000660 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000660 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_27.zip Uploading the intermediate file: upload_file_27.zip 12:47:08 STEP 673 H= 673:00 +CPU= 8.181 12:47:16 STEP 674 H= 674:00 +CPU= 8.207 12:47:42 STEP 675 H= 675:00 +CPU= 25.471 12:48:10 STEP 676 H= 676:00 +CPU= 27.807 12:48:41 STEP 677 H= 677:00 +CPU= 30.803 12:49:19 STEP 678 H= 678:00 +CPU= 38.140 12:49:51 STEP 679 H= 679:00 +CPU= 32.061 12:50:21 STEP 680 H= 680:00 +CPU= 30.077 12:51:07 STEP 681 H= 681:00 +CPU= 46.293 12:51:35 STEP 682 H= 682:00 +CPU= 27.355 12:51:48 STEP 683 H= 683:00 +CPU= 13.819 12:52:09 STEP 684 H= 684:00 +CPU= 20.332 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000684 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000684 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000684 12:52:19 STEP 685 H= 685:00 +CPU= 9.821 12:52:28 STEP 686 H= 686:00 +CPU= 9.740 12:52:43 STEP 687 H= 687:00 +CPU= 14.736 12:52:52 STEP 688 H= 688:00 +CPU= 8.957 12:53:00 STEP 689 H= 689:00 +CPU= 8.227 12:53:14 STEP 690 H= 690:00 +CPU= 13.235 12:53:26 STEP 691 H= 691:00 +CPU= 12.268 12:53:56 STEP 692 H= 692:00 +CPU= 30.254 12:54:45 STEP 693 H= 693:00 +CPU= 49.176 12:55:16 STEP 694 H= 694:00 +CPU= 30.867 12:55:46 STEP 695 H= 695:00 +CPU= 29.820 12:56:44 STEP 696 H= 696:00 +CPU= 58.069 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000696 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000696 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000696 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000672 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000672 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000672 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000684 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000684 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000684 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_28.zip Uploading the intermediate file: upload_file_28.zip 12:57:15 STEP 697 H= 697:00 +CPU= 30.454 12:57:42 STEP 698 H= 698:00 +CPU= 27.449 12:57:59 STEP 699 H= 699:00 +CPU= 17.046 12:58:10 STEP 700 H= 700:00 +CPU= 10.546 12:58:19 STEP 701 H= 701:00 +CPU= 9.611 12:58:34 STEP 702 H= 702:00 +CPU= 14.737 12:58:43 STEP 703 H= 703:00 +CPU= 8.675 12:58:51 STEP 704 H= 704:00 +CPU= 8.664 12:59:04 STEP 705 H= 705:00 +CPU= 13.206 12:59:13 STEP 706 H= 706:00 +CPU= 8.161 12:59:20 STEP 707 H= 707:00 +CPU= 7.851 12:59:35 STEP 708 H= 708:00 +CPU= 14.856 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000708 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000708 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000708 13:00:03 STEP 709 H= 709:00 +CPU= 27.741 13:00:36 STEP 710 H= 710:00 +CPU= 33.226 13:01:27 STEP 711 H= 711:00 +CPU= 50.687 13:01:53 STEP 712 H= 712:00 +CPU= 25.635 13:02:14 STEP 713 H= 713:00 +CPU= 21.085 13:02:47 STEP 714 H= 714:00 +CPU= 32.711 13:03:09 STEP 715 H= 715:00 +CPU= 22.240 13:03:31 STEP 716 H= 716:00 +CPU= 21.976 13:04:02 STEP 717 H= 717:00 +CPU= 31.544 13:04:14 STEP 718 H= 718:00 +CPU= 11.394 13:04:24 STEP 719 H= 719:00 +CPU= 10.529 13:04:42 STEP 720 H= 720:00 +CPU= 17.575 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000720 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000720 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000720 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000696 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000696 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000696 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000708 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000708 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000708 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_29.zip Uploading the intermediate file: upload_file_29.zip 13:04:51 STEP 721 H= 721:00 +CPU= 9.117 13:05:00 STEP 722 H= 722:00 +CPU= 8.787 Uploading trickle at timestep: 2592000 13:05:13 STEP 723 H= 723:00 +CPU= 13.581 13:05:22 STEP 724 H= 724:00 +CPU= 8.245 13:05:30 STEP 725 H= 725:00 +CPU= 7.897 13:06:12 STEP 726 H= 726:00 +CPU= 42.926 13:06:45 STEP 727 H= 727:00 +CPU= 32.382 13:07:18 STEP 728 H= 728:00 +CPU= 33.515 13:08:06 STEP 729 H= 729:00 +CPU= 47.567 13:08:30 STEP 730 H= 730:00 +CPU= 24.399 13:08:58 STEP 731 H= 731:00 +CPU= 28.013 13:09:47 STEP 732 H= 732:00 +CPU= 48.677 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000732 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000732 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000732 13:10:04 STEP 733 H= 733:00 +CPU= 16.989 13:10:16 STEP 734 H= 734:00 +CPU= 12.295 13:10:33 STEP 735 H= 735:00 +CPU= 17.086 13:10:44 STEP 736 H= 736:00 +CPU= 10.252 13:10:53 STEP 737 H= 737:00 +CPU= 9.020 13:11:08 STEP 738 H= 738:00 +CPU= 14.916 13:11:16 STEP 739 H= 739:00 +CPU= 8.409 13:11:25 STEP 740 H= 740:00 +CPU= 8.513 13:11:38 STEP 741 H= 741:00 +CPU= 13.218 13:11:46 STEP 742 H= 742:00 +CPU= 8.219 13:12:01 STEP 743 H= 743:00 +CPU= 15.393 13:12:48 STEP 744 H= 744:00 +CPU= 46.786 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000744 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000744 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000744 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000720 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000720 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000720 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000732 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000732 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000732 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_30.zip Uploading the intermediate file: upload_file_30.zip 13:13:08 STEP 745 H= 745:00 +CPU= 19.824 13:13:35 STEP 746 H= 746:00 +CPU= 27.102 13:14:09 STEP 747 H= 747:00 +CPU= 34.208 13:14:35 STEP 748 H= 748:00 +CPU= 25.873 13:15:03 STEP 749 H= 749:00 +CPU= 27.661 13:15:39 STEP 750 H= 750:00 +CPU= 36.597 13:16:05 STEP 751 H= 751:00 +CPU= 25.420 13:16:23 STEP 752 H= 752:00 +CPU= 18.286 13:16:43 STEP 753 H= 753:00 +CPU= 19.766 13:16:56 STEP 754 H= 754:00 +CPU= 12.847 13:17:07 STEP 755 H= 755:00 +CPU= 10.825 13:17:23 STEP 756 H= 756:00 +CPU= 16.609 13:17:32 STEP 757 H= 757:00 +CPU= 8.545 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000756 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000756 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000756 13:17:40 STEP 758 H= 758:00 +CPU= 8.602 13:17:54 STEP 759 H= 759:00 +CPU= 13.423 13:18:02 STEP 760 H= 760:00 +CPU= 8.353 13:18:10 STEP 761 H= 761:00 +CPU= 8.024 13:18:44 STEP 762 H= 762:00 +CPU= 34.273 13:19:13 STEP 763 H= 763:00 +CPU= 28.016 13:19:32 STEP 764 H= 764:00 +CPU= 19.116 13:20:10 STEP 765 H= 765:00 +CPU= 38.289 13:20:31 STEP 766 H= 766:00 +CPU= 20.690 13:20:55 STEP 767 H= 767:00 +CPU= 24.833 13:21:48 STEP 768 H= 768:00 +CPU= 52.953 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000768 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000768 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000768 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000744 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000744 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000744 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000756 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000756 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000756 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_31.zip Uploading the intermediate file: upload_file_31.zip 13:22:18 STEP 769 H= 769:00 +CPU= 29.608 13:22:45 STEP 770 H= 770:00 +CPU= 27.417 13:23:05 STEP 771 H= 771:00 +CPU= 19.730 13:23:16 STEP 772 H= 772:00 +CPU= 10.758 13:23:26 STEP 773 H= 773:00 +CPU= 9.860 13:23:41 STEP 774 H= 774:00 +CPU= 14.890 13:23:49 STEP 775 H= 775:00 +CPU= 8.496 13:23:58 STEP 776 H= 776:00 +CPU= 8.524 13:24:11 STEP 777 H= 777:00 +CPU= 13.101 13:24:20 STEP 778 H= 778:00 +CPU= 9.009 13:24:45 STEP 779 H= 779:00 +CPU= 24.899 13:25:33 STEP 780 H= 780:00 +CPU= 48.168 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000780 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000780 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000780 13:26:00 STEP 781 H= 781:00 +CPU= 26.845 13:26:28 STEP 782 H= 782:00 +CPU= 27.978 13:27:15 STEP 783 H= 783:00 +CPU= 47.674 13:27:45 STEP 784 H= 784:00 +CPU= 29.406 13:28:08 STEP 785 H= 785:00 +CPU= 23.033 13:28:47 STEP 786 H= 786:00 +CPU= 39.286 13:29:00 STEP 787 H= 787:00 +CPU= 12.867 13:29:11 STEP 788 H= 788:00 +CPU= 11.471 13:29:28 STEP 789 H= 789:00 +CPU= 16.611 13:29:38 STEP 790 H= 790:00 +CPU= 9.887 13:29:47 STEP 791 H= 791:00 +CPU= 9.177 13:30:03 STEP 792 H= 792:00 +CPU= 15.947 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000792 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000792 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000792 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000768 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000768 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000768 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000780 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000780 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000780 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_32.zip Uploading the intermediate file: upload_file_32.zip 13:30:11 STEP 793 H= 793:00 +CPU= 8.166 13:30:20 STEP 794 H= 794:00 +CPU= 8.291 13:30:32 STEP 795 H= 795:00 +CPU= 12.893 13:30:41 STEP 796 H= 796:00 +CPU= 8.155 13:30:48 STEP 797 H= 797:00 +CPU= 7.835 13:31:02 STEP 798 H= 798:00 +CPU= 13.762 13:31:20 STEP 799 H= 799:00 +CPU= 17.995 13:31:47 STEP 800 H= 800:00 +CPU= 26.357 13:32:28 STEP 801 H= 801:00 +CPU= 41.757 13:32:53 STEP 802 H= 802:00 +CPU= 24.452 13:33:25 STEP 803 H= 803:00 +CPU= 32.031 13:34:10 STEP 804 H= 804:00 +CPU= 44.998 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000804 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000804 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000804 13:34:36 STEP 805 H= 805:00 +CPU= 25.740 13:35:09 STEP 806 H= 806:00 +CPU= 33.727 13:35:36 STEP 807 H= 807:00 +CPU= 26.430 13:35:47 STEP 808 H= 808:00 +CPU= 11.382 13:35:57 STEP 809 H= 809:00 +CPU= 9.982 13:36:13 STEP 810 H= 810:00 +CPU= 15.432 13:36:21 STEP 811 H= 811:00 +CPU= 8.571 13:36:30 STEP 812 H= 812:00 +CPU= 8.413 13:36:43 STEP 813 H= 813:00 +CPU= 13.203 13:36:51 STEP 814 H= 814:00 +CPU= 8.173 13:37:00 STEP 815 H= 815:00 +CPU= 9.178 13:37:38 STEP 816 H= 816:00 +CPU= 37.556 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000816 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000816 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000816 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000792 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000792 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000792 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000804 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000804 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000804 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_33.zip Uploading the intermediate file: upload_file_33.zip 13:38:02 STEP 817 H= 817:00 +CPU= 24.504 13:38:29 STEP 818 H= 818:00 +CPU= 26.875 13:39:18 STEP 819 H= 819:00 +CPU= 48.773 13:39:47 STEP 820 H= 820:00 +CPU= 29.267 13:40:09 STEP 821 H= 821:00 +CPU= 21.748 13:40:52 STEP 822 H= 822:00 +CPU= 43.098 13:41:25 STEP 823 H= 823:00 +CPU= 32.803 13:41:39 STEP 824 H= 824:00 +CPU= 14.073 13:41:56 STEP 825 H= 825:00 +CPU= 16.453 13:42:05 STEP 826 H= 826:00 +CPU= 9.843 13:42:15 STEP 827 H= 827:00 +CPU= 9.244 13:42:30 STEP 828 H= 828:00 +CPU= 15.137 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000828 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000828 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000828 13:42:38 STEP 829 H= 829:00 +CPU= 8.140 13:42:46 STEP 830 H= 830:00 +CPU= 8.518 13:42:59 STEP 831 H= 831:00 +CPU= 12.935 13:43:15 STEP 832 H= 832:00 +CPU= 15.246 13:43:39 STEP 833 H= 833:00 +CPU= 24.436 13:44:20 STEP 834 H= 834:00 +CPU= 40.511 13:44:37 STEP 835 H= 835:00 +CPU= 17.508 13:45:00 STEP 836 H= 836:00 +CPU= 22.858 13:45:40 STEP 837 H= 837:00 +CPU= 40.192 13:46:14 STEP 838 H= 838:00 +CPU= 33.806 13:46:44 STEP 839 H= 839:00 +CPU= 29.535 13:47:41 STEP 840 H= 840:00 +CPU= 57.555 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000840 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000840 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000840 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000816 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000816 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000816 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000828 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000828 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000828 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_34.zip Uploading the intermediate file: upload_file_34.zip 13:47:56 STEP 841 H= 841:00 +CPU= 14.648 13:48:07 STEP 842 H= 842:00 +CPU= 10.898 13:48:23 STEP 843 H= 843:00 +CPU= 15.967 13:48:32 STEP 844 H= 844:00 +CPU= 9.370 13:48:41 STEP 845 H= 845:00 +CPU= 8.540 13:48:54 STEP 846 H= 846:00 +CPU= 13.923 13:49:03 STEP 847 H= 847:00 +CPU= 8.272 13:49:11 STEP 848 H= 848:00 +CPU= 8.191 13:49:43 STEP 849 H= 849:00 +CPU= 31.785 13:50:11 STEP 850 H= 850:00 +CPU= 28.135 13:50:37 STEP 851 H= 851:00 +CPU= 25.609 13:51:25 STEP 852 H= 852:00 +CPU= 48.603 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000852 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000852 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000852 13:51:55 STEP 853 H= 853:00 +CPU= 29.721 13:52:17 STEP 854 H= 854:00 +CPU= 21.555 13:53:08 STEP 855 H= 855:00 +CPU= 51.032 13:53:38 STEP 856 H= 856:00 +CPU= 29.751 13:53:54 STEP 857 H= 857:00 +CPU= 16.565 13:54:11 STEP 858 H= 858:00 +CPU= 17.361 13:54:21 STEP 859 H= 859:00 +CPU= 9.665 13:54:31 STEP 860 H= 860:00 +CPU= 9.419 13:54:45 STEP 861 H= 861:00 +CPU= 14.077 13:54:53 STEP 862 H= 862:00 +CPU= 8.556 13:55:02 STEP 863 H= 863:00 +CPU= 8.566 13:55:17 STEP 864 H= 864:00 +CPU= 15.640 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000864 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000864 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000864 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000840 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000840 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000840 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000852 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000852 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000852 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_35.zip Uploading the intermediate file: upload_file_35.zip 13:55:26 STEP 865 H= 865:00 +CPU= 8.125 13:55:34 STEP 866 H= 866:00 +CPU= 8.260 13:56:10 STEP 867 H= 867:00 +CPU= 36.189 13:56:39 STEP 868 H= 868:00 +CPU= 29.458 13:57:06 STEP 869 H= 869:00 +CPU= 26.361 13:57:48 STEP 870 H= 870:00 +CPU= 41.897 13:58:12 STEP 871 H= 871:00 +CPU= 24.737 13:58:40 STEP 872 H= 872:00 +CPU= 27.354 13:59:17 STEP 873 H= 873:00 +CPU= 37.543 13:59:47 STEP 874 H= 874:00 +CPU= 29.970 14:00:06 STEP 875 H= 875:00 +CPU= 18.607 14:00:26 STEP 876 H= 876:00 +CPU= 19.794 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000876 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000876 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000876 14:00:36 STEP 877 H= 877:00 +CPU= 10.432 14:00:46 STEP 878 H= 878:00 +CPU= 9.827 14:01:01 STEP 879 H= 879:00 +CPU= 14.613 14:01:10 STEP 880 H= 880:00 +CPU= 9.015 14:01:18 STEP 881 H= 881:00 +CPU= 8.429 14:01:32 STEP 882 H= 882:00 +CPU= 13.864 14:01:40 STEP 883 H= 883:00 +CPU= 8.275 14:01:49 STEP 884 H= 884:00 +CPU= 8.493 14:02:16 STEP 885 H= 885:00 +CPU= 26.757 14:02:43 STEP 886 H= 886:00 +CPU= 27.167 14:03:10 STEP 887 H= 887:00 +CPU= 27.027 14:03:58 STEP 888 H= 888:00 +CPU= 48.670 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000888 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000888 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000888 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000864 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000864 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000864 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000876 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000876 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000876 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_36.zip Uploading the intermediate file: upload_file_36.zip 14:04:27 STEP 889 H= 889:00 +CPU= 29.025 14:04:58 STEP 890 H= 890:00 +CPU= 30.970 14:05:34 STEP 891 H= 891:00 +CPU= 36.022 14:06:02 STEP 892 H= 892:00 +CPU= 27.916 14:06:23 STEP 893 H= 893:00 +CPU= 20.987 14:06:41 STEP 894 H= 894:00 +CPU= 18.014 14:06:52 STEP 895 H= 895:00 +CPU= 10.876 14:07:03 STEP 896 H= 896:00 +CPU= 10.305 14:07:18 STEP 897 H= 897:00 +CPU= 14.933 14:07:26 STEP 898 H= 898:00 +CPU= 8.947 14:07:35 STEP 899 H= 899:00 +CPU= 8.334 14:07:50 STEP 900 H= 900:00 +CPU= 14.788 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000900 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000900 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000900 14:07:58 STEP 901 H= 901:00 +CPU= 8.147 14:08:06 STEP 902 H= 902:00 +CPU= 8.358 14:08:38 STEP 903 H= 903:00 +CPU= 31.554 14:09:11 STEP 904 H= 904:00 +CPU= 33.373 14:09:43 STEP 905 H= 905:00 +CPU= 31.575 14:10:27 STEP 906 H= 906:00 +CPU= 44.579 14:10:52 STEP 907 H= 907:00 +CPU= 25.083 14:11:26 STEP 908 H= 908:00 +CPU= 33.745 14:12:07 STEP 909 H= 909:00 +CPU= 40.706 14:12:32 STEP 910 H= 910:00 +CPU= 25.116 14:12:44 STEP 911 H= 911:00 +CPU= 11.922 14:13:04 STEP 912 H= 912:00 +CPU= 20.352 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000912 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000912 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000912 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000888 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000888 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000888 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000900 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000900 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000900 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_37.zip Uploading the intermediate file: upload_file_37.zip 14:13:14 STEP 913 H= 913:00 +CPU= 9.520 14:13:23 STEP 914 H= 914:00 +CPU= 9.306 14:13:37 STEP 915 H= 915:00 +CPU= 13.894 14:13:46 STEP 916 H= 916:00 +CPU= 8.580 14:13:54 STEP 917 H= 917:00 +CPU= 8.210 14:14:07 STEP 918 H= 918:00 +CPU= 13.537 14:14:17 STEP 919 H= 919:00 +CPU= 9.751 14:14:42 STEP 920 H= 920:00 +CPU= 24.538 14:15:29 STEP 921 H= 921:00 +CPU= 47.002 14:15:57 STEP 922 H= 922:00 +CPU= 28.254 14:16:23 STEP 923 H= 923:00 +CPU= 26.398 14:17:18 STEP 924 H= 924:00 +CPU= 54.624 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000924 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000924 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000924 14:17:51 STEP 925 H= 925:00 +CPU= 33.557 14:18:22 STEP 926 H= 926:00 +CPU= 30.245 14:18:40 STEP 927 H= 927:00 +CPU= 17.974 14:18:51 STEP 928 H= 928:00 +CPU= 10.952 14:19:01 STEP 929 H= 929:00 +CPU= 10.214 14:19:17 STEP 930 H= 930:00 +CPU= 15.788 14:19:26 STEP 931 H= 931:00 +CPU= 9.205 14:19:35 STEP 932 H= 932:00 +CPU= 8.980 14:19:49 STEP 933 H= 933:00 +CPU= 13.642 14:19:57 STEP 934 H= 934:00 +CPU= 8.349 14:20:05 STEP 935 H= 935:00 +CPU= 8.228 14:20:37 STEP 936 H= 936:00 +CPU= 31.824 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000936 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000936 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000936 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000912 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000912 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000912 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000924 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000924 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000924 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_38.zip Uploading the intermediate file: upload_file_38.zip 14:21:10 STEP 937 H= 937:00 +CPU= 32.627 14:21:34 STEP 938 H= 938:00 +CPU= 24.424 14:22:20 STEP 939 H= 939:00 +CPU= 45.497 14:22:53 STEP 940 H= 940:00 +CPU= 33.715 14:23:26 STEP 941 H= 941:00 +CPU= 32.955 14:24:11 STEP 942 H= 942:00 +CPU= 44.738 14:24:22 STEP 943 H= 943:00 +CPU= 11.368 14:24:34 STEP 944 H= 944:00 +CPU= 11.704 14:24:52 STEP 945 H= 945:00 +CPU= 17.606 14:25:02 STEP 946 H= 946:00 +CPU= 10.595 14:25:12 STEP 947 H= 947:00 +CPU= 9.698 14:25:28 STEP 948 H= 948:00 +CPU= 16.429 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000948 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000948 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000948 14:25:37 STEP 949 H= 949:00 +CPU= 8.847 14:25:46 STEP 950 H= 950:00 +CPU= 8.694 14:25:59 STEP 951 H= 951:00 +CPU= 13.436 14:26:08 STEP 952 H= 952:00 +CPU= 8.382 14:26:24 STEP 953 H= 953:00 +CPU= 15.878 14:27:15 STEP 954 H= 954:00 +CPU= 51.696 14:27:47 STEP 955 H= 955:00 +CPU= 31.448 14:28:12 STEP 956 H= 956:00 +CPU= 25.352 14:28:49 STEP 957 H= 957:00 +CPU= 36.559 14:29:18 STEP 958 H= 958:00 +CPU= 28.781 14:29:49 STEP 959 H= 959:00 +CPU= 31.468 14:30:23 STEP 960 H= 960:00 +CPU= 33.942 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000960 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000960 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000960 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000936 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000936 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000936 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000948 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000948 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000948 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_39.zip Uploading the intermediate file: upload_file_39.zip 14:30:35 STEP 961 H= 961:00 +CPU= 12.448 Uploading trickle at timestep: 3456000 14:30:47 STEP 962 H= 962:00 +CPU= 11.746 14:31:04 STEP 963 H= 963:00 +CPU= 16.729 14:31:14 STEP 964 H= 964:00 +CPU= 10.454 14:31:24 STEP 965 H= 965:00 +CPU= 10.033 14:31:40 STEP 966 H= 966:00 +CPU= 15.397 14:31:48 STEP 967 H= 967:00 +CPU= 8.562 14:31:57 STEP 968 H= 968:00 +CPU= 8.737 14:32:11 STEP 969 H= 969:00 +CPU= 13.529 14:32:19 STEP 970 H= 970:00 +CPU= 8.271 14:32:27 STEP 971 H= 971:00 +CPU= 7.925 14:32:45 STEP 972 H= 972:00 +CPU= 18.581 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000972 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000972 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000972 14:33:04 STEP 973 H= 973:00 +CPU= 19.002 14:33:39 STEP 974 H= 974:00 +CPU= 34.591 14:34:20 STEP 975 H= 975:00 +CPU= 40.729 14:34:52 STEP 976 H= 976:00 +CPU= 32.518 14:35:15 STEP 977 H= 977:00 +CPU= 22.145 14:36:02 STEP 978 H= 978:00 +CPU= 47.018 14:36:26 STEP 979 H= 979:00 +CPU= 24.520 14:36:52 STEP 980 H= 980:00 +CPU= 25.319 14:37:14 STEP 981 H= 981:00 +CPU= 22.328 14:37:25 STEP 982 H= 982:00 +CPU= 11.388 14:37:36 STEP 983 H= 983:00 +CPU= 10.355 14:37:53 STEP 984 H= 984:00 +CPU= 17.387 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000984 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000984 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000984 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000960 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000960 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000960 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000972 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000972 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000972 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_40.zip Uploading the intermediate file: upload_file_40.zip 14:38:02 STEP 985 H= 985:00 +CPU= 8.524 14:38:10 STEP 986 H= 986:00 +CPU= 8.621 14:38:24 STEP 987 H= 987:00 +CPU= 13.463 14:38:32 STEP 988 H= 988:00 +CPU= 8.145 14:38:40 STEP 989 H= 989:00 +CPU= 7.834 14:38:53 STEP 990 H= 990:00 +CPU= 13.244 14:39:21 STEP 991 H= 991:00 +CPU= 27.717 14:39:54 STEP 992 H= 992:00 +CPU= 33.333 14:40:44 STEP 993 H= 993:00 +CPU= 50.193 14:41:14 STEP 994 H= 994:00 +CPU= 29.195 14:41:38 STEP 995 H= 995:00 +CPU= 24.463 14:42:19 STEP 996 H= 996:00 +CPU= 41.080 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+000996 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+000996 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+000996 14:42:44 STEP 997 H= 997:00 +CPU= 24.971 14:43:14 STEP 998 H= 998:00 +CPU= 29.601 14:43:31 STEP 999 H= 999:00 +CPU= 17.094 14:43:41 STEP 1000 H=1000:00 +CPU= 10.264 14:43:51 STEP 1001 H=1001:00 +CPU= 9.690 14:44:06 STEP 1002 H=1002:00 +CPU= 15.426 14:44:15 STEP 1003 H=1003:00 +CPU= 8.616 14:44:23 STEP 1004 H=1004:00 +CPU= 8.562 14:44:37 STEP 1005 H=1005:00 +CPU= 13.574 14:44:45 STEP 1006 H=1006:00 +CPU= 8.227 14:44:55 STEP 1007 H=1007:00 +CPU= 9.387 14:45:27 STEP 1008 H=1008:00 +CPU= 32.288 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+001008 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+001008 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+001008 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000984 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000984 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000984 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+000996 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+000996 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+000996 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_41.zip Uploading the intermediate file: upload_file_41.zip 14:45:56 STEP 1009 H=1009:00 +CPU= 28.725 14:46:28 STEP 1010 H=1010:00 +CPU= 32.044 14:47:20 STEP 1011 H=1011:00 +CPU= 52.213 14:47:50 STEP 1012 H=1012:00 +CPU= 29.585 14:48:20 STEP 1013 H=1013:00 +CPU= 29.981 14:49:03 STEP 1014 H=1014:00 +CPU= 42.987 14:49:24 STEP 1015 H=1015:00 +CPU= 21.097 14:49:36 STEP 1016 H=1016:00 +CPU= 12.255 14:49:52 STEP 1017 H=1017:00 +CPU= 16.587 14:50:02 STEP 1018 H=1018:00 +CPU= 10.032 14:50:12 STEP 1019 H=1019:00 +CPU= 9.324 14:50:27 STEP 1020 H=1020:00 +CPU= 15.220 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+001020 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+001020 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+001020 14:50:35 STEP 1021 H=1021:00 +CPU= 8.251 14:50:45 STEP 1022 H=1022:00 +CPU= 9.911 14:51:29 STEP 1023 H=1023:00 +CPU= 43.379 14:51:49 STEP 1024 H=1024:00 +CPU= 20.429 14:52:13 STEP 1025 H=1025:00 +CPU= 23.971 14:53:00 STEP 1026 H=1026:00 +CPU= 46.964 14:53:29 STEP 1027 H=1027:00 +CPU= 28.550 14:53:58 STEP 1028 H=1028:00 +CPU= 29.280 14:54:40 STEP 1029 H=1029:00 +CPU= 41.836 14:55:08 STEP 1030 H=1030:00 +CPU= 28.543 14:55:20 STEP 1031 H=1031:00 +CPU= 11.919 14:55:40 STEP 1032 H=1032:00 +CPU= 19.796 Moving to projects directory: /var/lib/41000/slots/3/ICMGGhnw4+001032 Moving to projects directory: /var/lib/41000/slots/3/ICMSHhnw4+001032 Moving to projects directory: /var/lib/41000/slots/3/ICMUAhnw4+001032 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+001008 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+001008 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+001008 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMGGhnw4+001020 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMSHhnw4+001020 Adding to the zip: /var/lib/41000/projects/climateprediction.net/oifs_43r3_12214609/ICMUAhnw4+001020 Zipping up the intermediate file: /var/lib/41000/projects/climateprediction.net/oifs_43r3_00wj_2019110100_123_993_12214609_0_r1771899607_42.zip Uploading the intermediate file: upload_file_42.zip 14:55:50 STEP 1033 H=1033:00 +CPU= 9.816 14:55:59 STEP 1034 H=1034:00 +CPU= 9.498 14:56:14 STEP 1035 H=1035:00 +CPU= 14.245 double free or corruption (!prev) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 16:52:52 | 1539146 | 22315982 | oifs_43r3_00wj_2019110100_123_993_12214609_0 | 3,456,000 | 20,656 | 0.0060 |
09 May 2023 16:52:49 | 1539146 | 22315982 | oifs_43r3_00wj_2019110100_123_993_12214609_0 | 2,592,000 | 15,521 | 0.0060 |
09 May 2023 16:52:45 | 1539146 | 22315982 | oifs_43r3_00wj_2019110100_123_993_12214609_0 | 1,728,000 | 10,398 | 0.0060 |
09 May 2023 16:52:43 | 1539146 | 22315982 | oifs_43r3_00wj_2019110100_123_993_12214609_0 | 864,000 | 5,284 | 0.0061 |
©2024 climateprediction.net