climateprediction.net home page

large work unit?


Advanced search

Message boards : Number crunching : large work unit?

AuthorMessage
VJ
Send message
Joined: Nov 5 04
Posts: 19
Credit: 88,724
RAC: 0
Message 6222 - Posted 19 Nov 2004 7:50:41 UTC

    Hello,

    I noticed something strange with the workunit size.

    I\'m using one computer that doesn\'t have network, and another computer that does. The latter one is also used to download work/upload data for the non-networked computer (I\'m working on a tutorial on how to set this up :))

    Now, the workunit on my networked computer is estimated to take 600 hours (over halfway now), but the workunits I downloaded for my offline system are estimated to take 1600 hours.
    This is even the case when I move those workunits to the networked computer, so it doesn\'t seem to be related to the computer speed (they are both Intel at 2.4 GHz anyway).

    The main difference in the download was that for downloading the workunits for the offline system, I specified a long time between network access (this default value is 0.1 time per day).
    Did this cause it?


    Jörg

    HOAP
    Send message
    Joined: Sep 3 04
    Posts: 3
    Credit: 44,416
    RAC: 24
    Message 6229 - Posted 19 Nov 2004 9:46:11 UTC - in response to Message 6222.

      The estimated time to completion is based on the benchmark scores.

      Have you checked if your "identical" systems have identical scores?

      There is a known bug when benchmarking multiple CPU (or HT) systems causing the benchmark scores to fluctuate greatly. If your computer is HT then this might be the cause.


      > Hello,
      >
      > I noticed something strange with the workunit size.
      >
      > I'm using one computer that doesn't have network, and another computer that
      > does. The latter one is also used to download work/upload data for the
      > non-networked computer (I'm working on a tutorial on how to set this up :))
      >
      > Now, the workunit on my networked computer is estimated to take 600 hours
      > (over halfway now), but the workunits I downloaded for my offline system are
      > estimated to take 1600 hours.
      > This is even the case when I move those workunits to the networked computer,
      > so it doesn't seem to be related to the computer speed (they are both Intel at
      > 2.4 GHz anyway).
      >
      > The main difference in the download was that for downloading the workunits for
      > the offline system, I specified a long time between network access (this
      > default value is 0.1 time per day).
      > Did this cause it?
      >
      >
      > Jörg
      >

      VJ
      Send message
      Joined: Nov 5 04
      Posts: 19
      Credit: 88,724
      RAC: 0
      Message 6230 - Posted 19 Nov 2004 10:17:47 UTC - in response to Message 6229.

        > The estimated time to completion is based on the benchmark scores.
        >
        > Have you checked if your "identical" systems have identical scores?

        They are close. One is a P4 2.4 GHz, the other a dual Xeon 2.4 GHz (hyperthreading enabled). For the dual Xeon, it shows the benchmark scores per logical CPU and they seem to more or less match the P4 benchmarks.

        The Xeon is doing 4 units in parallel, each with an estimated time of 1600 hours, the elapsed time does change with real time (as does the cpu time spent on the different processes). However, when I move the work from the Xeon to the other computer, it only runs one unit (which is logical) but gives the same time estimate. The Xeon has been running for 160 hours, and the graphs are at 10%, so it is not only the estimate that is wrong: the unit does take longer to work on.

        > There is a known bug when benchmarking multiple CPU (or HT) systems causing
        > the benchmark scores to fluctuate greatly. If your computer is HT then this
        > might be the cause.

        Yes, but that wouldn't explain why the calculations also take longer...? Would it?


        Jörg

        Toby
        Send message
        Joined: Aug 17 04
        Posts: 56
        Credit: 63,814
        RAC: 0
        Message 6253 - Posted 20 Nov 2004 2:49:44 UTC

          Might be helpful if you didn't hide your computers as we could look at the numbers ourselves... However your processing times don't surprise me in the least. Hyperthreading does NOT give you the processing power of 2 CPUs. Depending on what type of project you are running, it will yield different results. Projects that hit the main system RAM a lot will take more of a performance hit on HT processors than projects that can fit a good chunk of their data in the L2 cache. Since CPDN takes 50 MB of RAM, I would suspect that it would fall under the former category. Furthermore even dual CPU setups don't give you 2x the performance for many of the same reasons. The memory bus often turns into a bottleneck.

          If you were running them 1 by 1, 4 work units would take you 2,400 hours. Completing all 4 of them in 1,600 on a dual HT machine seems very reasonable to me.

          VJ
          Send message
          Joined: Nov 5 04
          Posts: 19
          Credit: 88,724
          RAC: 0
          Message 6333 - Posted 24 Nov 2004 7:49:46 UTC

            It was the hyperthreading...

            Running 2 units in parallel shows an estimate time of 600 hours for the 2 units that are paused. The 2 running units show the same estimated time, but it runs faster. :-)

            Still, from 1600 hours for 4 units in parallel to 600 hours for 2 units in parallel (thus 1200 hours for all 4 of them) is quite a performance increase.


            Jörg
            ____________

            Post to thread

            Message boards : Number crunching : large work unit?




            Copyright © 2002-2014 climateprediction.net