climateprediction.net home page
Task 22275273

Task 22275273

Name oifs_43r3_ps_0078_2004050100_123_973_12189722_0
Workunit 12189722
Created 23 Dec 2022, 7:01:47 UTC
Sent 30 Dec 2022, 16:59:30 UTC
Report deadline 29 Jan 2023, 16:59:30 UTC
Received 4 Jan 2023, 17:26:59 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1529817
Run time 13 hours 36 min 56 sec
CPU time 12 hours 30 min 31 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 4.72 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,416.18 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,689.24 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ps_0078_2004050100_123_973_12189722.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
ic_ancil_file: ic_ancil_12189722
ifsdata_file: ifsdata_12189722
climate_data_file: clim_data_12189722
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6f47685f1aaa5667d26260bcb220e59b to: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_80c62163f69d97e714ad12f01b148a3d to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12189722.xml
-- Model is restarting --
Adjusting last_iter, 471, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
last_cpu_time: 33055
upload_file_number: 19
last_iter: 455
last_upload: 1641600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717
The child process has been launched with process id: 25703
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:25703:25703] [20230104:135854:1672840734:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  14:00:03 STEP  456 H= 456:00 +CPU= 61.745
  14:00:38 STEP  457 H= 457:00 +CPU= 35.206
  14:01:14 STEP  458 H= 458:00 +CPU= 35.440
  14:02:08 STEP  459 H= 459:00 +CPU= 53.499
  14:02:40 STEP  460 H= 460:00 +CPU= 31.854
  14:03:13 STEP  461 H= 461:00 +CPU= 31.941
  14:04:09 STEP  462 H= 462:00 +CPU= 55.455
  14:04:44 STEP  463 H= 463:00 +CPU= 35.219
  14:05:20 STEP  464 H= 464:00 +CPU= 35.710
  14:06:12 STEP  465 H= 465:00 +CPU= 51.226
  14:06:44 STEP  466 H= 466:00 +CPU= 32.053
  14:07:19 STEP  467 H= 467:00 +CPU= 33.883
  14:08:19 STEP  468 H= 468:00 +CPU= 59.248
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000468
  14:08:51 STEP  469 H= 469:00 +CPU= 31.613
  14:09:25 STEP  470 H= 470:00 +CPU= 33.789
  14:10:22 STEP  471 H= 471:00 +CPU= 56.181
  14:10:57 STEP  472 H= 472:00 +CPU= 35.334
  14:11:31 STEP  473 H= 473:00 +CPU= 33.222
  14:12:25 STEP  474 H= 474:00 +CPU= 52.562
  14:13:03 STEP  475 H= 475:00 +CPU= 36.332
  14:13:41 STEP  476 H= 476:00 +CPU= 37.059
  14:14:41 STEP  477 H= 477:00 +CPU= 56.533
  14:15:14 STEP  478 H= 478:00 +CPU= 32.390
  14:15:50 STEP  479 H= 479:00 +CPU= 35.645
  14:16:58 STEP  480 H= 480:00 +CPU= 65.328
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_19.zip
Uploading the intermediate file: upload_file_19.zip
  14:17:34 STEP  481 H= 481:00 +CPU= 35.865
  14:18:12 STEP  482 H= 482:00 +CPU= 36.752
  14:19:05 STEP  483 H= 483:00 +CPU= 52.314
  14:19:39 STEP  484 H= 484:00 +CPU= 32.490
  14:20:15 STEP  485 H= 485:00 +CPU= 35.730
  14:21:13 STEP  486 H= 486:00 +CPU= 56.221
  14:21:45 STEP  487 H= 487:00 +CPU= 31.298
  14:22:19 STEP  488 H= 488:00 +CPU= 34.163
  14:23:16 STEP  489 H= 489:00 +CPU= 55.484
  14:23:51 STEP  490 H= 490:00 +CPU= 35.468
  14:24:28 STEP  491 H= 491:00 +CPU= 35.718
  14:25:26 STEP  492 H= 492:00 +CPU= 57.598
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000492
  14:25:58 STEP  493 H= 493:00 +CPU= 31.942
  14:26:35 STEP  494 H= 494:00 +CPU= 36.011
  14:27:28 STEP  495 H= 495:00 +CPU= 53.191
  14:28:01 STEP  496 H= 496:00 +CPU= 32.330
  14:28:35 STEP  497 H= 497:00 +CPU= 33.468
  14:29:32 STEP  498 H= 498:00 +CPU= 56.224
  14:30:05 STEP  499 H= 499:00 +CPU= 32.513
  14:30:38 STEP  500 H= 500:00 +CPU= 32.781
  14:31:35 STEP  501 H= 501:00 +CPU= 55.680
  14:32:11 STEP  502 H= 502:00 +CPU= 35.662
  14:32:43 STEP  503 H= 503:00 +CPU= 30.935
  14:33:42 STEP  504 H= 504:00 +CPU= 58.309
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_20.zip
Uploading the intermediate file: upload_file_20.zip
  14:34:12 STEP  505 H= 505:00 +CPU= 30.078
  14:34:48 STEP  506 H= 506:00 +CPU= 35.910
  14:35:45 STEP  507 H= 507:00 +CPU= 55.998
  14:36:24 STEP  508 H= 508:00 +CPU= 37.217
  14:37:03 STEP  509 H= 509:00 +CPU= 36.695
  14:37:59 STEP  510 H= 510:00 +CPU= 54.640
  14:38:33 STEP  511 H= 511:00 +CPU= 32.495
  14:39:12 STEP  512 H= 512:00 +CPU= 37.654
  14:40:07 STEP  513 H= 513:00 +CPU= 53.768
  14:40:39 STEP  514 H= 514:00 +CPU= 32.046
  14:41:14 STEP  515 H= 515:00 +CPU= 34.134
  14:42:16 STEP  516 H= 516:00 +CPU= 60.903
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000516
  14:42:52 STEP  517 H= 517:00 +CPU= 35.374
  14:43:28 STEP  518 H= 518:00 +CPU= 36.237
  14:44:22 STEP  519 H= 519:00 +CPU= 52.245
  14:44:57 STEP  520 H= 520:00 +CPU= 33.868
  14:45:32 STEP  521 H= 521:00 +CPU= 34.887
  14:46:27 STEP  522 H= 522:00 +CPU= 54.604
  14:47:00 STEP  523 H= 523:00 +CPU= 31.744
  14:47:35 STEP  524 H= 524:00 +CPU= 35.152
  14:48:32 STEP  525 H= 525:00 +CPU= 55.919
  14:49:11 STEP  526 H= 526:00 +CPU= 37.641
  14:49:51 STEP  527 H= 527:00 +CPU= 37.720
  14:50:56 STEP  528 H= 528:00 +CPU= 62.441
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_21.zip
Uploading the intermediate file: upload_file_21.zip
  14:51:34 STEP  529 H= 529:00 +CPU= 34.905
  14:52:11 STEP  530 H= 530:00 +CPU= 36.577
  14:53:07 STEP  531 H= 531:00 +CPU= 54.950
  14:53:41 STEP  532 H= 532:00 +CPU= 33.452
  14:54:13 STEP  533 H= 533:00 +CPU= 31.548
  14:55:10 STEP  534 H= 534:00 +CPU= 55.829
  14:55:46 STEP  535 H= 535:00 +CPU= 35.318
  14:56:23 STEP  536 H= 536:00 +CPU= 36.303
  14:57:19 STEP  537 H= 537:00 +CPU= 54.335
  14:57:53 STEP  538 H= 538:00 +CPU= 33.011
  14:58:29 STEP  539 H= 539:00 +CPU= 34.774
  14:59:35 STEP  540 H= 540:00 +CPU= 63.359
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000540
  15:00:15 STEP  541 H= 541:00 +CPU= 36.928
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0078
(argv4) batchid: 973
(argv5) wuid: 12189722
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_56adf6388a421a5d6e548c2950b2df44 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
ic_ancil_file: ic_ancil_12189722
ifsdata_file: ifsdata_12189722
climate_data_file: clim_data_12189722
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6f47685f1aaa5667d26260bcb220e59b to: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_80c62163f69d97e714ad12f01b148a3d to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12189722.xml
-- Model is restarting --
Adjusting last_iter, 541, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
last_cpu_time: 36654
upload_file_number: 22
last_iter: 527
last_upload: 1900800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717
The child process has been launched with process id: 31616
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:31616:31616] [20230104:150129:1672844489:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:02:38 STEP  528 H= 528:00 +CPU= 63.375
  15:03:15 STEP  529 H= 529:00 +CPU= 35.637
  15:03:50 STEP  530 H= 530:00 +CPU= 33.876
  15:04:42 STEP  531 H= 531:00 +CPU= 51.070
  15:05:18 STEP  532 H= 532:00 +CPU= 35.388
  15:05:53 STEP  533 H= 533:00 +CPU= 35.191
  15:06:50 STEP  534 H= 534:00 +CPU= 55.698
  15:07:21 STEP  535 H= 535:00 +CPU= 31.478
  15:07:55 STEP  536 H= 536:00 +CPU= 32.873
  15:08:50 STEP  537 H= 537:00 +CPU= 54.899
  15:09:27 STEP  538 H= 538:00 +CPU= 36.579
  15:10:04 STEP  539 H= 539:00 +CPU= 35.927
  15:11:03 STEP  540 H= 540:00 +CPU= 58.355
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000540
  15:11:34 STEP  541 H= 541:00 +CPU= 30.572
  15:12:10 STEP  542 H= 542:00 +CPU= 36.465
  15:13:05 STEP  543 H= 543:00 +CPU= 54.305
  15:13:37 STEP  544 H= 544:00 +CPU= 30.769
  15:14:10 STEP  545 H= 545:00 +CPU= 33.018
  15:15:08 STEP  546 H= 546:00 +CPU= 57.405
  15:15:44 STEP  547 H= 547:00 +CPU= 35.153
  15:16:21 STEP  548 H= 548:00 +CPU= 36.690
  15:17:14 STEP  549 H= 549:00 +CPU= 52.097
  15:17:46 STEP  550 H= 550:00 +CPU= 31.752
  15:18:21 STEP  551 H= 551:00 +CPU= 35.346
  15:19:26 STEP  552 H= 552:00 +CPU= 63.951
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_22.zip
Uploading the intermediate file: upload_file_22.zip
  15:20:02 STEP  553 H= 553:00 +CPU= 34.928
  15:20:37 STEP  554 H= 554:00 +CPU= 34.822
  15:21:28 STEP  555 H= 555:00 +CPU= 50.152
  15:22:04 STEP  556 H= 556:00 +CPU= 35.732
  15:22:39 STEP  557 H= 557:00 +CPU= 35.024
  15:23:37 STEP  558 H= 558:00 +CPU= 57.098
  15:24:13 STEP  559 H= 559:00 +CPU= 35.083
  15:24:46 STEP  560 H= 560:00 +CPU= 32.867
  15:25:38 STEP  561 H= 561:00 +CPU= 51.624
  15:26:15 STEP  562 H= 562:00 +CPU= 35.771
  15:26:50 STEP  563 H= 563:00 +CPU= 35.330
  15:27:51 STEP  564 H= 564:00 +CPU= 59.487
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000564
  15:28:22 STEP  565 H= 565:00 +CPU= 30.745
  15:28:58 STEP  566 H= 566:00 +CPU= 35.613
  15:29:54 STEP  567 H= 567:00 +CPU= 55.819
  15:30:31 STEP  568 H= 568:00 +CPU= 36.132
  15:31:08 STEP  569 H= 569:00 +CPU= 36.465
  15:32:02 STEP  570 H= 570:00 +CPU= 53.628
  15:32:33 STEP  571 H= 571:00 +CPU= 30.871
  15:33:09 STEP  572 H= 572:00 +CPU= 34.878
  15:34:05 STEP  573 H= 573:00 +CPU= 55.242
  15:34:41 STEP  574 H= 574:00 +CPU= 36.259
  15:35:15 STEP  575 H= 575:00 +CPU= 32.913
  15:36:15 STEP  576 H= 576:00 +CPU= 59.518
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_23.zip
Uploading the intermediate file: upload_file_23.zip
  15:36:50 STEP  577 H= 577:00 +CPU= 34.268
  15:37:26 STEP  578 H= 578:00 +CPU= 36.160
  15:38:24 STEP  579 H= 579:00 +CPU= 56.884
  15:38:57 STEP  580 H= 580:00 +CPU= 33.201
  15:39:29 STEP  581 H= 581:00 +CPU= 30.931
  15:40:26 STEP  582 H= 582:00 +CPU= 56.151
  15:40:59 STEP  583 H= 583:00 +CPU= 33.234
  15:41:35 STEP  584 H= 584:00 +CPU= 35.082
  15:42:29 STEP  585 H= 585:00 +CPU= 53.917
  15:43:02 STEP  586 H= 586:00 +CPU= 31.819
  15:43:34 STEP  587 H= 587:00 +CPU= 32.466
  15:44:38 STEP  588 H= 588:00 +CPU= 62.613
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000588
  15:45:12 STEP  589 H= 589:00 +CPU= 33.579
  15:45:44 STEP  590 H= 590:00 +CPU= 32.151
  15:46:38 STEP  591 H= 591:00 +CPU= 52.630
  15:47:14 STEP  592 H= 592:00 +CPU= 35.885
  15:47:49 STEP  593 H= 593:00 +CPU= 34.791
  15:48:46 STEP  594 H= 594:00 +CPU= 55.928
  15:49:19 STEP  595 H= 595:00 +CPU= 32.798
  15:49:52 STEP  596 H= 596:00 +CPU= 32.956
  15:50:48 STEP  597 H= 597:00 +CPU= 54.733
  15:51:24 STEP  598 H= 598:00 +CPU= 35.721
  15:52:00 STEP  599 H= 599:00 +CPU= 35.732
  15:53:03 STEP  600 H= 600:00 +CPU= 62.495
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_24.zip
Uploading the intermediate file: upload_file_24.zip
  15:53:35 STEP  601 H= 601:00 +CPU= 31.081
  15:54:10 STEP  602 H= 602:00 +CPU= 34.843
  15:55:07 STEP  603 H= 603:00 +CPU= 56.430
  15:55:43 STEP  604 H= 604:00 +CPU= 35.140
  15:56:14 STEP  605 H= 605:00 +CPU= 31.039
  15:57:10 STEP  606 H= 606:00 +CPU= 54.827
  15:57:45 STEP  607 H= 607:00 +CPU= 34.277
  15:58:22 STEP  608 H= 608:00 +CPU= 36.236
  15:59:18 STEP  609 H= 609:00 +CPU= 55.440
  15:59:52 STEP  610 H= 610:00 +CPU= 33.860
  16:00:24 STEP  611 H= 611:00 +CPU= 31.229
  16:01:24 STEP  612 H= 612:00 +CPU= 59.136
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000612
  16:01:59 STEP  613 H= 613:00 +CPU= 35.328
  16:02:36 STEP  614 H= 614:00 +CPU= 36.334
  16:03:32 STEP  615 H= 615:00 +CPU= 55.420
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0078
(argv4) batchid: 973
(argv5) wuid: 12189722
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_56adf6388a421a5d6e548c2950b2df44 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
ic_ancil_file: ic_ancil_12189722
ifsdata_file: ifsdata_12189722
climate_data_file: clim_data_12189722
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6f47685f1aaa5667d26260bcb220e59b to: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_80c62163f69d97e714ad12f01b148a3d to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12189722.xml
-- Model is restarting --
Adjusting last_iter, 616, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
last_cpu_time: 40334
upload_file_number: 25
last_iter: 599
last_upload: 2160000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717
The child process has been launched with process id: 36023
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:36023:36023] [20230104:160449:1672848289:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:05:55 STEP  600 H= 600:00 +CPU= 61.976
  16:06:32 STEP  601 H= 601:00 +CPU= 35.660
  16:07:07 STEP  602 H= 602:00 +CPU= 34.546
  16:07:57 STEP  603 H= 603:00 +CPU= 49.800
  16:08:32 STEP  604 H= 604:00 +CPU= 34.713
  16:09:07 STEP  605 H= 605:00 +CPU= 34.368
  16:10:06 STEP  606 H= 606:00 +CPU= 56.444
  16:10:37 STEP  607 H= 607:00 +CPU= 30.540
  16:11:11 STEP  608 H= 608:00 +CPU= 33.299
  16:12:03 STEP  609 H= 609:00 +CPU= 51.777
  16:12:36 STEP  610 H= 610:00 +CPU= 32.280
  16:13:12 STEP  611 H= 611:00 +CPU= 35.246
  16:14:20 STEP  612 H= 612:00 +CPU= 64.190
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000612
  16:14:55 STEP  613 H= 613:00 +CPU= 34.256
  16:15:29 STEP  614 H= 614:00 +CPU= 33.168
  16:16:25 STEP  615 H= 615:00 +CPU= 54.630
  16:17:02 STEP  616 H= 616:00 +CPU= 36.706
  16:17:35 STEP  617 H= 617:00 +CPU= 32.264
  16:18:31 STEP  618 H= 618:00 +CPU= 53.976
  16:19:08 STEP  619 H= 619:00 +CPU= 35.601
  16:19:46 STEP  620 H= 620:00 +CPU= 37.100
  16:20:42 STEP  621 H= 621:00 +CPU= 55.442
  16:21:20 STEP  622 H= 622:00 +CPU= 36.744
  16:21:58 STEP  623 H= 623:00 +CPU= 35.981
  16:22:57 STEP  624 H= 624:00 +CPU= 58.937
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_25.zip
Uploading the intermediate file: upload_file_25.zip
  16:23:33 STEP  625 H= 625:00 +CPU= 34.984
  16:24:12 STEP  626 H= 626:00 +CPU= 37.617
  16:25:10 STEP  627 H= 627:00 +CPU= 56.731
  16:25:46 STEP  628 H= 628:00 +CPU= 35.724
  16:26:20 STEP  629 H= 629:00 +CPU= 33.008
  16:27:12 STEP  630 H= 630:00 +CPU= 51.760
  16:27:48 STEP  631 H= 631:00 +CPU= 35.216
  16:28:24 STEP  632 H= 632:00 +CPU= 35.621
  16:29:18 STEP  633 H= 633:00 +CPU= 53.512
  16:29:51 STEP  634 H= 634:00 +CPU= 32.156
  16:30:23 STEP  635 H= 635:00 +CPU= 32.227
  16:31:25 STEP  636 H= 636:00 +CPU= 61.293
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000636
  16:32:02 STEP  637 H= 637:00 +CPU= 35.891
  16:32:40 STEP  638 H= 638:00 +CPU= 37.092
  16:33:37 STEP  639 H= 639:00 +CPU= 54.854
  16:34:09 STEP  640 H= 640:00 +CPU= 31.971
  16:34:41 STEP  641 H= 641:00 +CPU= 31.686
  16:35:39 STEP  642 H= 642:00 +CPU= 56.859
  16:36:14 STEP  643 H= 643:00 +CPU= 34.559
  16:36:46 STEP  644 H= 644:00 +CPU= 31.762
  16:37:39 STEP  645 H= 645:00 +CPU= 52.494
  16:38:15 STEP  646 H= 646:00 +CPU= 35.728
  16:38:50 STEP  647 H= 647:00 +CPU= 34.010
  16:39:48 STEP  648 H= 648:00 +CPU= 58.059
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_26.zip
Uploading the intermediate file: upload_file_26.zip
  16:40:24 STEP  649 H= 649:00 +CPU= 34.820
  16:41:00 STEP  650 H= 650:00 +CPU= 36.370
  16:41:57 STEP  651 H= 651:00 +CPU= 55.732
  16:42:33 STEP  652 H= 652:00 +CPU= 35.721
  16:43:07 STEP  653 H= 653:00 +CPU= 33.187
  16:44:05 STEP  654 H= 654:00 +CPU= 54.856
  16:44:43 STEP  655 H= 655:00 +CPU= 36.453
  16:45:25 STEP  656 H= 656:00 +CPU= 39.049
  16:46:26 STEP  657 H= 657:00 +CPU= 57.479
  16:47:01 STEP  658 H= 658:00 +CPU= 34.633
  16:47:32 STEP  659 H= 659:00 +CPU= 30.816
  16:48:34 STEP  660 H= 660:00 +CPU= 60.274
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000660
  16:49:10 STEP  661 H= 661:00 +CPU= 35.565
  16:49:46 STEP  662 H= 662:00 +CPU= 34.899
  16:50:39 STEP  663 H= 663:00 +CPU= 51.577
  16:51:16 STEP  664 H= 664:00 +CPU= 36.131
  16:51:52 STEP  665 H= 665:00 +CPU= 35.179
  16:52:46 STEP  666 H= 666:00 +CPU= 53.477
  16:53:21 STEP  667 H= 667:00 +CPU= 32.872
  16:53:59 STEP  668 H= 668:00 +CPU= 37.396
  16:54:55 STEP  669 H= 669:00 +CPU= 54.714
  16:55:28 STEP  670 H= 670:00 +CPU= 32.096
  16:56:03 STEP  671 H= 671:00 +CPU= 34.113
  16:57:07 STEP  672 H= 672:00 +CPU= 64.061
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_27.zip
Uploading the intermediate file: upload_file_27.zip
  16:57:43 STEP  673 H= 673:00 +CPU= 35.171
  16:58:20 STEP  674 H= 674:00 +CPU= 36.523
  16:59:16 STEP  675 H= 675:00 +CPU= 55.591
  16:59:52 STEP  676 H= 676:00 +CPU= 35.805
  17:00:24 STEP  677 H= 677:00 +CPU= 31.192
  17:01:19 STEP  678 H= 678:00 +CPU= 54.772
  17:01:55 STEP  679 H= 679:00 +CPU= 35.749
  17:02:30 STEP  680 H= 680:00 +CPU= 34.241
  17:03:20 STEP  681 H= 681:00 +CPU= 49.608
  17:03:56 STEP  682 H= 682:00 +CPU= 35.779
  17:04:32 STEP  683 H= 683:00 +CPU= 35.384
  17:05:37 STEP  684 H= 684:00 +CPU= 63.317
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000684
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0078
(argv4) batchid: 973
(argv5) wuid: 12189722
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/4
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_56adf6388a421a5d6e548c2950b2df44 to: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/4/oifs_43r3_ps_0078_2004050100_123_973_12189722.zip
ic_ancil_file: ic_ancil_12189722
ifsdata_file: ifsdata_12189722
climate_data_file: clim_data_12189722
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6f47685f1aaa5667d26260bcb220e59b to: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/4/ic_ancil_12189722.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_80c62163f69d97e714ad12f01b148a3d to: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/4/ifsdata/ifsdata_12189722.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/4/159l_2/clim_data_12189722.zip
Checking for progress XML file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
Opened progress file ok : /var/lib/boinc-client/slots/4/progress_file_12189722.xml
-- Model is restarting --
Adjusting last_iter, 685, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/4/progress_file_12189722.xml
last_cpu_time: 43927
upload_file_number: 28
last_iter: 671
last_upload: 2419200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717
The child process has been launched with process id: 40422
Executing the command: /var/lib/boinc-client/slots/4/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:cyberpunk:1:1:40422:40422] [20230104:170707:1672852027:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  17:08:14 STEP  672 H= 672:00 +CPU= 62.830
  17:08:47 STEP  673 H= 673:00 +CPU= 32.646
  17:09:20 STEP  674 H= 674:00 +CPU= 32.294
  17:10:15 STEP  675 H= 675:00 +CPU= 54.933
  17:10:52 STEP  676 H= 676:00 +CPU= 36.199
  17:11:30 STEP  677 H= 677:00 +CPU= 35.860
  17:12:30 STEP  678 H= 678:00 +CPU= 58.349
  17:13:03 STEP  679 H= 679:00 +CPU= 32.503
  17:13:36 STEP  680 H= 680:00 +CPU= 32.228
  17:14:32 STEP  681 H= 681:00 +CPU= 54.984
  17:15:09 STEP  682 H= 682:00 +CPU= 36.542
  17:15:45 STEP  683 H= 683:00 +CPU= 35.611
  17:16:44 STEP  684 H= 684:00 +CPU= 58.869
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000684
  17:17:16 STEP  685 H= 685:00 +CPU= 31.553
  17:17:52 STEP  686 H= 686:00 +CPU= 34.902
  17:18:47 STEP  687 H= 687:00 +CPU= 55.000
  17:19:23 STEP  688 H= 688:00 +CPU= 35.331
  17:19:56 STEP  689 H= 689:00 +CPU= 32.233
  17:20:48 STEP  690 H= 690:00 +CPU= 51.966
  17:21:24 STEP  691 H= 691:00 +CPU= 35.612
  17:22:01 STEP  692 H= 692:00 +CPU= 36.169
  17:22:55 STEP  693 H= 693:00 +CPU= 54.035
  17:23:28 STEP  694 H= 694:00 +CPU= 32.002
  17:24:02 STEP  695 H= 695:00 +CPU= 33.825
  17:25:06 STEP  696 H= 696:00 +CPU= 63.563
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/4/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189722/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0078_2004050100_123_973_12189722_0_r786768717_28.zip
Uploading the intermediate file: upload_file_28.zip
  17:25:42 STEP  697 H= 697:00 +CPU= 34.998
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
No trickles!


©2024 cpdn.org