climateprediction.net (CPDN) home page
Thread 'Credit inconsistancy.'

Thread 'Credit inconsistancy.'

Message boards : Number crunching : Credit inconsistancy.
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54954 - Posted: 18 Oct 2016, 13:50:13 UTC
Last modified: 18 Oct 2016, 14:07:34 UTC

I mentioned in another thread recently a problem with suspending and restarting jobs. I have been waiting for the last of my cpdn work units to finish, which it now has. I looked at my results page to check everything was good with results from the last few days. What I saw distrubed me.

>>>
20027986 10726123 1411044 13 Oct 2016 16:39:03 UTC 18 Oct 2016 13:03:57 UTC Completed 247,734.29 244,198.20 779.66 779.66 Weather At Home 2 (wah2) v8.12
20027182 10751049 1411044 13 Oct 2016 6:48:05 UTC 17 Oct 2016 6:04:18 UTC Completed 211,985.74 208,492.50 3,059.47 3,059.47 Weather At Home 2 (wah2) v8.12
20024387 10756708 1411044 12 Oct 2016 19:05:39 UTC 16 Oct 2016 21:36:27 UTC Completed 231,223.01 226,950.60 5,339.28 5,339.28 Weather At Home 2 (wah2) v8.12
20022060 10754457 1411044 12 Oct 2016 18:01:21 UTC 16 Oct 2016 18:08:22 UTC Completed 233,385.97 229,230.70 5,339.28 5,339.28 Weather At Home 2 (wah2) v8.12
20021964 10754361 1411044 12 Oct 2016 18:01:21 UTC 16 Oct 2016 7:30:22 UTC Error while computing 215,353.16 211,354.10 5,339.28 5,339.28 Weather At Home 2 (wah2) v8.12
20019882 10752351 1411044 12 Oct 2016 21:05:17 UTC 17 Oct 2016 0:23:39 UTC Completed 227,234.29 223,338.40 4,579.34 4,579.34 Weather At Home 2 (wah2) v8.12
<<<

So, jobs that run for days sometimes get thousands of credits, whilst here, the longest running of those gets 779.66. Even the job that crashed when I restarted them got 5,339.28.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54954 · Report as offensive     Reply Quote
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,904,898
RAC: 2,026
Message 54959 - Posted: 18 Oct 2016, 17:02:47 UTC - in response to Message 54954.  

The credit script runs once a week, so you need to wait until the credit run after the model finishes before starting to worry that something has gone wrong. Up until that point your models will have only some of the credits that will eventually be allocated.
ID: 54959 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54961 - Posted: 18 Oct 2016, 18:34:50 UTC

Fair enough.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54961 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54973 - Posted: 20 Oct 2016, 19:57:44 UTC

Whilst we are waiting for the credit adjustment of those results, lets have a look at these...

>>>
19691493 10594489 1392227 4 Jun 2016 4:31:21 UTC 22 Jun 2016 10:14:04 UTC Completed 419,595.89 413,064.60 18,258.20 18,258.20 Weather At Home 2 (wah2) v8.12
19652717 10564965 1392227 26 May 2016 13:30:53 UTC 28 Jun 2016 10:47:15 UTC Completed 668,194.80 656,637.20 9,898.90 9,898.90 Weather At Home 2 (wah2) v8.12
19636470 10550587 1392227 4 Jun 2016 8:06:01 UTC 21 Jul 2016 8:00:30 UTC Completed 594,126.85 580,855.90 9,898.90 9,898.90 Weather At Home 2 (wah2) v8.12
19634177 10548422 1392227 4 Jun 2016 14:08:08 UTC 23 Jul 2016 2:23:52 UTC Completed 573,682.269 562,921.40 9,898.90 9,898.90 Weather At Home 2 (wah2) v8.12
<<<
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54973 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54974 - Posted: 20 Oct 2016, 20:52:02 UTC - in response to Message 54973.  

Hi adrianxw

Additionally, you need to look at the individual model pages.

The first one is a pnw25, batch 406, which ran for 24 months.
The second is an eu25, batch 403, which ran for 13 months.
The last two are eu25, batch 400, which ran for 13 months.

For 24 months, the credit is 18258.20
divide by 24 = 760.758 per month

For 13 months, the credit is 9898.90
divide by 13 = 761.4538 per month

Which is fairly close.

Is this what you mean?

ID: 54974 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54987 - Posted: 24 Oct 2016, 10:02:15 UTC

What I mean, simply put, is that cpdn used my machine for periods of time and granted credit, apparently, on other criteria. Frankly, it makes no difference to me what the job was doing, what matters is that it was using my systems, thus preventing other BOINC projects from using them, for periods, usually extended periods, of time.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54987 · Report as offensive     Reply Quote
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,904,898
RAC: 2,026
Message 54988 - Posted: 24 Oct 2016, 12:58:55 UTC
Last modified: 24 Oct 2016, 13:02:57 UTC

The CPDN credit system has not kept up with developments in its applications. Applications used to be released to cover a model type or region, whole-world vs Europe, for example: since credits are determined on an application basis, credits could then be awarded that reflected the time taken to run that particular type of model. For understandable reasons there is now a smaller number of generic applications, such as WAH2, whose run-time character is determined by downloaded data files (including region) and not by the application itself. The project has retained the run length in months as a parameter for scaling the award of credits but, as your examples show, these don't always reflect the actual run-times very well. Perhaps there could be a per-batch credit rate - but batches can be really quite small and the project might not know how long they take to run until they've been run generally. You can just imagine the complaints if a batch credit rate were to be reduced after release!

Someone who crunches across BOINC projects might have a more sophisticated view than me but, as far as I can tell, the introduction of GPU applications returning huge credits pretty much destroyed the incentive to harmonise cross-project CPU credit rates.
ID: 54988 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54989 - Posted: 24 Oct 2016, 14:57:37 UTC

The effect I am seeing is the credit granted to work units of the same application at the same project. GPU work is not a factor here either.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54989 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54995 - Posted: 24 Oct 2016, 20:42:18 UTC - in response to Message 54989.  

To paraphrase what Iain said, application names here can no longer be used as a measure of anything.
Or, as Shakespeare said: Aye, there's the rub.

This all changed with the move to the wah application, which is region independent, and run time independent. Larger hi-res regions require longer processing time compared to a small region. (pnw area is small, South America is big).
And the resolution can be varied as well. At present they seem to be using just two: 25Km, and 50Km.

A recent batch was for only 3 months, (which could be run in a few hours), up to, at present, 24 months. And MUCH longer runs could be issued.
All with the one application.

ID: 54995 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54996 - Posted: 25 Oct 2016, 14:50:36 UTC

So what you are saying is the effects are to be expected and correct.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54996 · Report as offensive     Reply Quote
Les Bayliss
Volunteer moderator

Send message
Joined: 5 Sep 04
Posts: 7629
Credit: 24,240,330
RAC: 0
Message 54997 - Posted: 26 Oct 2016, 5:15:43 UTC - in response to Message 54996.  

Yes, that's it.

ID: 54997 · Report as offensive     Reply Quote
Profileadrianxw
Avatar

Send message
Joined: 31 Aug 04
Posts: 145
Credit: 2,080,724
RAC: 753
Message 54999 - Posted: 26 Oct 2016, 7:04:58 UTC
Last modified: 26 Oct 2016, 7:37:07 UTC

It is screamingly obvious to me, simply from the results I have quoted here, that the way they are running things is not right. I have been with the project for a long time, but sadly, I have crunched my last cpdn work unit here. They are using a resource, but fail to understand its dynamics. Detatching. I hope they learn something from that.
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
ID: 54999 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 492
Credit: 31,523,112
RAC: 14,304
Message 55000 - Posted: 26 Oct 2016, 8:53:49 UTC - in response to Message 54988.  

Do you know if there is a relationship between the gflops for a run and the credit awarded? The longer runs are larger in this respect.
ID: 55000 · Report as offensive     Reply Quote
bernard_ivo

Send message
Joined: 18 Jul 13
Posts: 438
Credit: 25,837,810
RAC: 9,716
Message 55001 - Posted: 26 Oct 2016, 12:12:28 UTC - in response to Message 55000.  

Perhaps Gflops related credit would be more fair.

For example wah2_eu25_13 months run on my machines more than 400 hours and gets the same credit as wah2_mex50_13 which runs 2.5 times faster. As far as I can get this discrepancy was part of the message of adrianxw and I share this concern. Though I have no intention to detach perhaps it is an issue for many crunchers - current, future or past.
ID: 55001 · Report as offensive     Reply Quote
Dave Roberts

Send message
Joined: 15 Jan 11
Posts: 175
Credit: 6,242,691
RAC: 699
Message 55003 - Posted: 26 Oct 2016, 19:05:57 UTC

So there has been a significant change in the way the scientists are devising models of the climate, in terms of the basic ontology of work units.
With the credit system devised for the older model types, perhaps we now have an example of "unforeseen effects"?

If the change is permanent, either in total or part, I guess we'll have to wait it out.

It's taking part that counts. If I'd been developing new model types/model structures, I'm sure it wouldn't have occurred to me to think about knock-on effects to credits in BOINC.

ID: 55003 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 492
Credit: 31,523,112
RAC: 14,304
Message 55006 - Posted: 27 Oct 2016, 10:58:10 UTC - in response to Message 55001.  
Last modified: 27 Oct 2016, 11:00:16 UTC

Curiously I have recently had varying amounts of credit for CAM50 tasks on the same computer which all appear to be the same length producing the same number of trickles. Wierd? It could be a credit run timing thing.
ID: 55006 · Report as offensive     Reply Quote
ProfileIain Inglis
Volunteer moderator

Send message
Joined: 16 Jan 10
Posts: 1084
Credit: 7,904,898
RAC: 2,026
Message 55007 - Posted: 27 Oct 2016, 11:41:19 UTC - in response to Message 55006.  

Curiously I have recently had varying amounts of credit for CAM50 tasks on the same computer which all appear to be the same length producing the same number of trickles. Wierd? It could be a credit run timing thing.

Most likely. The way I check up on the credit run timings is to look at the "statistics" tab in BOINC Manager, then "Show user average" - that will display a staircase of credit allocations and indicate when the next run is expected.
ID: 55007 · Report as offensive     Reply Quote
ProfileAlan K

Send message
Joined: 22 Feb 06
Posts: 492
Credit: 31,523,112
RAC: 14,304
Message 55026 - Posted: 28 Oct 2016, 17:56:42 UTC - in response to Message 55007.  

Credit oddities sorted after todays run. Hooray!!
ID: 55026 · Report as offensive     Reply Quote
ed2353

Send message
Joined: 15 Feb 06
Posts: 137
Credit: 35,517,114
RAC: 10,523
Message 55031 - Posted: 29 Oct 2016, 9:03:53 UTC - in response to Message 55026.  
Last modified: 29 Oct 2016, 9:05:31 UTC

However, I think there is another oddity. I suspect that the double recording of Trickles is also doubling our credit!
ID: 55031 · Report as offensive     Reply Quote
Profilegeophi
Volunteer moderator

Send message
Joined: 7 Aug 04
Posts: 2187
Credit: 64,822,615
RAC: 5,275
Message 55036 - Posted: 29 Oct 2016, 14:47:51 UTC - in response to Message 55031.  

However, I think there is another oddity. I suspect that the double recording of Trickles is also doubling our credit!

I think we're fine. This task, for example, has the "proper" amount of credits, yet has a trickle listed twice. And it completed before the credit script was run.
ID: 55036 · Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Credit inconsistancy.

©2024 cpdn.org