climateprediction.net home page
large work unit?

large work unit?

Message boards : Number crunching : large work unit?
Message board moderation

To post messages, you must log in.

AuthorMessage
old_user28601

Send message
Joined: 5 Nov 04
Posts: 19
Credit: 88,724
RAC: 0
Message 6222 - Posted: 19 Nov 2004, 7:50:41 UTC

Hello,

I noticed something strange with the workunit size.

I\'m using one computer that doesn\'t have network, and another computer that does. The latter one is also used to download work/upload data for the non-networked computer (I\'m working on a tutorial on how to set this up :))

Now, the workunit on my networked computer is estimated to take 600 hours (over halfway now), but the workunits I downloaded for my offline system are estimated to take 1600 hours.
This is even the case when I move those workunits to the networked computer, so it doesn\'t seem to be related to the computer speed (they are both Intel at 2.4 GHz anyway).

The main difference in the download was that for downloading the workunits for the offline system, I specified a long time between network access (this default value is 0.1 time per day).
Did this cause it?


Jörg
ID: 6222 · Report as offensive     Reply Quote
HOAP

Send message
Joined: 3 Sep 04
Posts: 3
Credit: 44,416
RAC: 0
Message 6229 - Posted: 19 Nov 2004, 9:46:11 UTC - in response to Message 6222.  

The estimated time to completion is based on the benchmark scores.

Have you checked if your "identical" systems have identical scores?

There is a known bug when benchmarking multiple CPU (or HT) systems causing the benchmark scores to fluctuate greatly. If your computer is HT then this might be the cause.


> Hello,
>
> I noticed something strange with the workunit size.
>
> I'm using one computer that doesn't have network, and another computer that
> does. The latter one is also used to download work/upload data for the
> non-networked computer (I'm working on a tutorial on how to set this up :))
>
> Now, the workunit on my networked computer is estimated to take 600 hours
> (over halfway now), but the workunits I downloaded for my offline system are
> estimated to take 1600 hours.
> This is even the case when I move those workunits to the networked computer,
> so it doesn't seem to be related to the computer speed (they are both Intel at
> 2.4 GHz anyway).
>
> The main difference in the download was that for downloading the workunits for
> the offline system, I specified a long time between network access (this
> default value is 0.1 time per day).
> Did this cause it?
>
>
> Jörg
>
ID: 6229 · Report as offensive     Reply Quote
old_user28601

Send message
Joined: 5 Nov 04
Posts: 19
Credit: 88,724
RAC: 0
Message 6230 - Posted: 19 Nov 2004, 10:17:47 UTC - in response to Message 6229.  

> The estimated time to completion is based on the benchmark scores.
>
> Have you checked if your "identical" systems have identical scores?

They are close. One is a P4 2.4 GHz, the other a dual Xeon 2.4 GHz (hyperthreading enabled). For the dual Xeon, it shows the benchmark scores per logical CPU and they seem to more or less match the P4 benchmarks.

The Xeon is doing 4 units in parallel, each with an estimated time of 1600 hours, the elapsed time does change with real time (as does the cpu time spent on the different processes). However, when I move the work from the Xeon to the other computer, it only runs one unit (which is logical) but gives the same time estimate. The Xeon has been running for 160 hours, and the graphs are at 10%, so it is not only the estimate that is wrong: the unit does take longer to work on.

> There is a known bug when benchmarking multiple CPU (or HT) systems causing
> the benchmark scores to fluctuate greatly. If your computer is HT then this
> might be the cause.

Yes, but that wouldn't explain why the calculations also take longer...? Would it?


Jörg
ID: 6230 · Report as offensive     Reply Quote
old_user909

Send message
Joined: 17 Aug 04
Posts: 56
Credit: 63,814
RAC: 0
Message 6253 - Posted: 20 Nov 2004, 2:49:44 UTC

Might be helpful if you didn't hide your computers as we could look at the numbers ourselves... However your processing times don't surprise me in the least. Hyperthreading does NOT give you the processing power of 2 CPUs. Depending on what type of project you are running, it will yield different results. Projects that hit the main system RAM a lot will take more of a performance hit on HT processors than projects that can fit a good chunk of their data in the L2 cache. Since CPDN takes 50 MB of RAM, I would suspect that it would fall under the former category. Furthermore even dual CPU setups don't give you 2x the performance for many of the same reasons. The memory bus often turns into a bottleneck.

If you were running them 1 by 1, 4 work units would take you 2,400 hours. Completing all 4 of them in 1,600 on a dual HT machine seems very reasonable to me.
ID: 6253 · Report as offensive     Reply Quote
old_user28601

Send message
Joined: 5 Nov 04
Posts: 19
Credit: 88,724
RAC: 0
Message 6333 - Posted: 24 Nov 2004, 7:49:46 UTC

It was the hyperthreading...

Running 2 units in parallel shows an estimate time of 600 hours for the 2 units that are paused. The 2 running units show the same estimated time, but it runs faster. :-)

Still, from 1600 hours for 4 units in parallel to 600 hours for 2 units in parallel (thus 1200 hours for all 4 of them) is quite a performance increase.


Jörg
ID: 6333 · Report as offensive     Reply Quote

Message boards : Number crunching : large work unit?

©2024 climateprediction.net