Message boards :
Number crunching :
Credit not synchronized on stats-pages
Message board moderation
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
Send message Joined: 12 Sep 04 Posts: 34 Credit: 1,017,702 RAC: 0 |
As they have access to the required information, I would think the project admins are in a better position to perform the necessary process of elimination. Could they not take the credits reported on their database and those on any one of the external sites and do a v-lookup in excel or run a simple script to determine which users are affected and then revert to their own data to determine what these users have in common? |
Send message Joined: 3 Oct 06 Posts: 43 Credit: 8,017,057 RAC: 0 |
Sometime in December I had to renew My Cross Project ID on the BOINCStats site. Could the problem be related to that? Currently The BOINCStats site shows my credits correctly for CPDN, as far as I can tell. ;) |
Send message Joined: 12 Sep 04 Posts: 34 Credit: 1,017,702 RAC: 0 |
Sometime in December I had to renew My Cross Project ID on the BOINCStats site. Could the problem be related to that? Currently The BOINCStats site shows my credits correctly for CPDN, as far as I can tell. ;) This is not the cause: 1. It is not limited to BoincStats. All external stats sites record the same for CPDN. 2. My user number both here and on these sites is consistently 17104. 3. My cross-project ID is the same on all external stats sites. I do not see why this would affect the credit for just this one project. Somehow, there is a breakdown in the completeness of the data ending up on these sites and it would appear to be not limited to just me. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
There was a problem late last year caused by the mirroring server set up, as I think, was explained in a News post at the time. This was fixed late December. This was the cause of the original poster's original problem. This new problem is different. I'll pass on a message. Please keep in mind that: A) The project people may still be on leave, and B) They aren't being paid by the various research groups to spend all day every day fiddling with credit problems. So be patient. It may take a while. Backups: Here |
Send message Joined: 22 Sep 11 Posts: 24 Credit: 243,945 RAC: 0 |
Hi Les, etal, On your recommendation I inquired via email with an admin over at one of the stats sites. I described the issue with them and referred them to this thread and their take on the problem was this...
...Anyhoo, hope this is of some help for you folks! :) Jimmy G |
Send message Joined: 12 Sep 04 Posts: 34 Credit: 1,017,702 RAC: 0 |
There was a problem late last year caused by the mirroring server set up, as I think, was explained in a News post at the time. This was fixed late December. Thanks Les. I understand that they have more important issues to deal with, as long as it gets looked at sometime. |
Send message Joined: 22 Sep 11 Posts: 24 Credit: 243,945 RAC: 0 |
To Whom It May Concern, Perhaps someone at CPDN can drop the folks over at MWAH a line about how they went about solving their recent stat site credit reporting issue?... Free DC does not report MW credit: http://milkyway.cs.rpi.edu/milkyway/forum_thread.php?id=2739 ...Just trying to put some problem-solvers together. :) Jimmy G |
Send message Joined: 11 Mar 08 Posts: 5 Credit: 13,296,444 RAC: 0 |
I see during the last weeks constantly a difference in the number of credits on my account page and the number reported in the project overview at the bottom of the account view. So it looks like the number as exported to the stat sites is missing about 20k credits. |
Send message Joined: 22 Sep 11 Posts: 24 Credit: 243,945 RAC: 0 |
Hi etrecords, FWIW, since my previous post back in January I tried a few things to see if the credits would catch up on the stats pages...all to no avail. I checked with a stat board admin (as suggested earlier in this thread by CPDN folks) to see if the problem was on their end and, as my resulting post showed, the problem is with CPDN. I temporarily stopped processing existing CPDN WUs for a week or so to see if there was some sort of reporting lag, and that didn't solve the discrepancy. I tried completing all my CPDN WUs and then waiting to see if the credits caught up, but that didn't solve anything, either. So, here I am (going on two months later) with all the stats sites reporting the same 231,504 for CPDN when the actual number is 243,945...a missing 12,441 credits or, roughly, 5% of my CPDN total...all accumulated in just 4 months of running their software... :( And we're not alone on this missing credit thing. I've randomly checked other user stats and, while not universal, there are clearly other folks experiencing this problem. Anyhoo, that's where things seem to be at this point from where I'm sitting. I wish I had more encouraging news. Best to you, :) |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
The problem lies with the last attempt to sync the main database and the mirror database just before Christmas. This was only partly successful, with some credits still missing. The project people know about this. Backups: Here |
Send message Joined: 12 Sep 04 Posts: 34 Credit: 1,017,702 RAC: 0 |
Glacial progress? Warped |
Send message Joined: 22 Sep 11 Posts: 24 Credit: 243,945 RAC: 0 |
...3-month check-in/reminder...same numbers, no change... |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
Credits aren't high on the Too Do list. They're probably way beyond the local event horizon by now. Backups: Here |
Send message Joined: 22 Sep 11 Posts: 24 Credit: 243,945 RAC: 0 |
Credits aren't high on the Too Do list. They're probably way beyond the local event horizon by now. Hi Les, I appreciate your candor. It's unfortunate that user credit-issues and accurate member-data reporting are not on management's radar, it doesn't make for a good storefront window-dressing for their project, imho. Oh well. I'll keep checking back to see how things are from time-to-time, in the meantime my machines will continue to be busy with other projects. Best, as always, :) |
Send message Joined: 15 May 09 Posts: 4472 Credit: 18,448,326 RAC: 22,385 |
In an ideal world I would agree. However in an ideal world there would be no need for CPDN. As it is, the funding for this project means that they have to prioritise. I for one would rather see the time spent on keeping the hardware running and enabling a few more thousand work units to be sent out and processed rather than the time and money being used to sort out stats. problems. The science of this project is to me far more important than my own or anyone else's ranking in league tables of how many cpu cycles they have contributed. Dave |
Send message Joined: 15 Jan 11 Posts: 175 Credit: 6,242,691 RAC: 699 |
I agree totally with Dave, I'd run CPDN work without any credit system at all and any 'window dressing' should be the last thing to tackle. The results from CPDN work might just make the difference in international economic/environmental decision making. Lets face it, if the climate system goes on the rampage (however one wants to describe worst, or even medium, case scenarios), everything else will fade into insignificance anyway! OK, public credit is always nice but people all over the world give to charities with no thought of recognition. Friends of mine who run CPDN work don't even bother to look at the stats page. Nuff said?? David Roberts |
Send message Joined: 22 Sep 11 Posts: 24 Credit: 243,945 RAC: 0 |
Hi Dave and David, etal, I appreciate your altruistic views here, truly and sincerely. Now, I could go into a whole long drawn-out bit about how I voluntarily pledged to myself to give up both owning and using an internal-combustion-engine automobile for a whole year (and am now into my second year of doing so) and try to impress the world with my real-life acts of consumer greening, but I'll restrain myself here. In fact, it was because of my beliefs and views on the often detrimental impacts of human activity on the global environment that I joined CPDN...I thought it a positive way to help offset the impact of the coal I was burning to generate the electricity to run my machines. All that aside, we're all participating in BOINC projects. IMHO, this should be a fun activity for one-and-all. In the course of participation user-issues arise. In this case, the accurate and timely reporting of accumulated imaginary credits. (I know, pathetic, really.) And here at CPDN it has been an ongoing issue for the several months I've been here, and the issue, clearly, predates me, as well. Oh dear, what's a user to do?! :) Some will say nothing and just keep crunching along hoping a fix will magically appear one day...this is the non-dev-feedback approach to systems problem-solving and usually results in years of stuff not working properly. Some will bring it to the attention of the powers-that-be (who may or may not know a problem exists) in the hopes of a resolution...this is the positive-dev-feedback approach to ironing out bugs, glitches and other system misbehaviors, and it usually results in things getting fixed. I just finished up 32 years in my telecom career and I've got to tell you, I'm a "second-kinda-guy" of the two because that's how things get done. :) The forums are CPDNs storefront in my analogy above, and what the customers see is a project with data-handling issues. "Hmm, if they can't produce consistent and accurate low-level user-data needs how well are their high-level climate-data needs being handled?!" (Yeah, I know, I'm using my "If the tables at your local restaurant are dirty, one can be fairly sure that the kitchen is not any better." analogy here...sue me.) And for a project like CPDN, in this environment of "climate change poo-pooing", I should think they'd like to allay any concerns of their data's accuracy, consistency and handling. Of course, all of this is just my humble opinion. Now, in the real world of telecoms with paying clients the average is about 5-out-of-every-100 customers with a service-affecting problem will generate a call-in complaint. And with the advent of customer feedback forums the telecoms discovered that many types of service-related complaints actually went down...a phenomenon of "when one sees that someone else has already filed an identical complaint they feel it's already been reported and the problem will be resolved"...it doesn't mean there are less over-all problems, necessarily, it just means it's harder to judge how wide-spread the problem is based on this smaller sampling...and so, companies (well, those who care about their customers) have had to learn to pay closer attention to their customer's feedback data. Mine is a long-form service complaint. :) So, just what is my long-form post all about? Simple, I'd like to see CPDN be successful! As of today's stats CPDN has an installed user base of 260,440 folks with an active rate of 19,702, and it has a host-base of 529,508 machines with only 26,769 active. From here... Overall project stats for Climateprediction.net: http://stats.kwsn.net/project.php?proj=cpdn ...and one can easily determine for their own pleasure the long-term user participation trends from the graphs found here... Boinc all Project Stats: http://allprojectstats.com/po.php?projekt=21 ...and I'll be crazy and go out on a limb here and speculate that some of the folks not participating or who have defected have chosen to do so because of the (wait for it...here it comes) credit issue. The only difference between them and me is, they chose not to tell anyone that's why they stopped! So, I'll be down the street at the place with the clean forks if you need me. Oh yeah, I'll be taking a stroll to the grocery store later, can I bring you back anything?! Best, as always! :) |
Send message Joined: 15 May 09 Posts: 4472 Credit: 18,448,326 RAC: 22,385 |
Hi Jimmy, I have no doubt that you are correct in your assumption that you are not the only person t leave CPDN because of this issue. It would be nice for the project people to at least issue an apology for the problem and a time scale for looking at it if they can. I know from 30 years working in the health service that an apology often does a lot to keep people on board. I see looking at the server stats page that one of the servers is down again. Given the problems they have had with the servers over the past year, I can also understand that this is a higher priority as those uploading to these servers will not get any credits at all unless they are fixed, indeed the whole crunching for the project becomes a total waste of time if the results can not be uploaded. So I am not knocking your points, just saying that there are other priorities. Hope you are back with the project soon. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
I'll try and keep this short. It was decided long before BOINC showed up, to send data back to the servers at intervals through the long models. And each of these trickles was translated into credits as per the SETI system, and on a similar basis to the then SETI system. Then BOINC showed up, and the project was modified to use that. But BOINC insisted on granting credits at the END of all processing. So the credits were granted twice. BUT ... only for some people. Cue loud screams. So the credits system was changed to an inhouse system - when a new trickle showed up for a computer, ALL trickles for that computer were counted, and then the total was converted into credit. Cue image of the pages of a calendar turning rapidly. But suddenly this process alone was taking up large amounts of server time. So it was changed a bit and made a 'chron job', running twice a day. And after a year or so, once per day. And still it was taking a long time, and slowing down the server. (I think 6-8 hours at the end.) So, another change. Archive the data of completed models up to that point, and create a 'correction factor' of credits for each computer/user/team, which could be added to future credits. With several hundred terabytes of data, not as easy as it sounded. And this has now been done twice, each time with problems. The current problem goes back to late last year, when a mirror system was introduced for the server with this board plus the user's credits, etc. This was because the server had slowed to the point were it was inaccessible for 12 or more hours each day, due to the credits program, backups, and researchers running search programs on the data. The plan was to run the credits program on the mirror server, freeing up the main server for the other tasks. There were problems, the end result of which was that not all of the credits on the mirror were added to the various totals. And that's where we stand at present. What's needed is a keen, enthusiastic person with lots of time on their hands, to volunteer to go through the backups and mirrors, and get a list of all of the billions of trickles, along with their associated model type, and then translate that to credits. Finally, the two project people are being paid by the research groups around the world to get their programs working reliably, and to then make these available to the public, together with data files, for creating the climate models wanted by these researchers. Backups: Here |
Send message Joined: 15 May 09 Posts: 4472 Credit: 18,448,326 RAC: 22,385 |
Thanks Les, it is good to have a fuller picture than I could without trawling through lots of old posts. I look at the stats pages on the Everest principle - Because they are there but would still crunch for the project if they did not exist. Clearly the project could do with an injection of some serious cash. |
©2024 cpdn.org