# FAST team stats... announcemnt



## cine.chris (Feb 23, 2021)

Announcement, but not like an Nvidia or AMD announcement.
It's actually working, currently tweaking the pieces to get the pages updated on the webserver
Sample from yesterday's EOD end-of-day:
TPU sample page
Rank shown is your True Active Project Rank, with abandons removed.
Note: this is different from Production Rank which is usually a 7 day average of point production.
I'll post an update with the live link soon.
The current project stats, with the new NEW FAH servers, are updating hourly.
So, expect some glitches with all the new moving pieces.
Yes, it needs a team index.
Anyway, you guys are the first to see it!


----------



## phill (Feb 23, 2021)

Great work there @cine.chris !!


----------



## cine.chris (Feb 23, 2021)

Let me know if you are seeing updates, ~ on the hour...


			Team Data


----------



## Jacky_BEL (Feb 24, 2021)

Seems to work OK.


----------



## cine.chris (Feb 25, 2021)

Jacky_BEL said:


> Seems to work OK.


Yes, except, adding to their other server problems their downloads/timestamps are now in a constant state of drift & appear to not be linked to a cron process.
Like yesterday, their update drifted past the final update. 
My post on the Folding Forums got no responses so, I guess that nobody is concerned with the science of stats... just folding proteins.
I've now shifted my cron task 25 minutes since starting this project and it will cross the hour, which forced me to shuffle the zero hour.
My only option is to update every 15 minutes until this is fixed.
Fortunately, my process is efficient, running in ~2min, which considering that you are dealing with two sets of 2.5M records is impressive. 
The credit goes to Python & Pandas, not my crude skill set.
Anyway... the page is beautiful & fast on a phone.  No tracking, no ads, no click-bait swill, no graphics, just pure HTML & Jquery.

Rev2 image from my phone, which will be live soon!


----------



## Jacky_BEL (Feb 25, 2021)

I have noticed on the https://apps.foldingathome.org/cpu recent CPU page that reporting the "last returned" seems to happen every 10 minutes, at the 06 16 26 36 46 and 56th minute.


----------



## cine.chris (Feb 27, 2021)

The FAH updates have stalled... Uggh.
They could be returning to a 3hr update rate or just broken, I'll have to wait & see.
Lot's of problems with their infrastructure lately...


----------



## Jacky_BEL (Feb 27, 2021)

Yes, some of my machines have errors in the log file, trying to contact the collection server.
All machines are now running though.
But the F@H recent CPU page has stopped updating also.

Timestamp on the last F@h Fast Teampage URL is Sat Feb 27 04:04:26 GMT 2021
and on my F@h donor page it is 2021-02-27 04:47:21

I hope maybe the DC-stats page now gets a reset somehow too 

Results are available again.


----------



## phill (Feb 27, 2021)

It seems that Extreme and Free DC had an issue today..  Normally it's just Free-DC...  That seems to never work 100% well for FAH...  WCG and Rosetta seems to work ok tho....  FAH not even close


----------



## Jacky_BEL (Feb 27, 2021)

The app from @cine.chris seems to suffer the same error as DC-stats right now with exagerated numbers.
Maybe the reporting timing issue as Chris stated.

Anyway, I guess it is no simple task to track all those folding results.


----------



## cine.chris (Feb 28, 2021)

That's what it should do with no EOD end-of-day stats for the prior day.  (by-design)
So, it shows the two days combined in this case.
Will Jacky_BEL make 30Million ???
I also set the EOD an hour earlier, which is US EST.
As, EOD is a synthetic construct anyway.  I've always thought it should be GMT.
When everything is working correctly, there will be a collection of Team EOD files at:
http://fahtech.com/EOD/ , but not working yet.


----------



## phill (Feb 28, 2021)

I'd consider using the Extreme Forum one, I think that one actually does update properly for the day and I've not noticed it being down for more than a day but that does tend to have a lot more accurate information, it just doesn't show milestones sadly....


----------



## cine.chris (Feb 28, 2021)

Folding data all comes from the same source. 
With the shifting download update time, now moving at 3.6 min/day.  There is a likely hood that people could be pulling different data.  As they only provide cumulative totals, not hourly or daily, it's the same data everywhere.  The key variable now is how the floating update time is managed & tracked.  My tract is to follow that point and update data when the data changes and not follow the clock, as the next update could be 1 min away & missed.
Of course, one hour slightly off is still much better than every 3 hrs.
My current validation is to make sure that the end/start data between days matches & no data is lost in the first hour stats.
Again, for everyone, the final data is what Folding says it is.  It's all the same data.... if downloaded at the same time.
That's one reason I'll be logging the EOD stats.  Right now, everything is essentially a chalkboard and erased in a few days.
Currently, I have published RANK data that goes back to 20190101.


			Index of /rank
		

Try & find data that really describes the behavior of the Folding Project over the years... it doesn't exist.
Currently, I have data back to Mid-2013, so more historical perspectives are what I want to provide and a framework to understand how group projects like this behave.


----------



## Jacky_BEL (Feb 28, 2021)

Now it all starts to make more sense to me.
EOC seem to have found a good workaround, but it only updates every 3 hours.

I have bookmarked the app in my browser


----------



## cine.chris (Feb 28, 2021)

With the restart, the stats are still drifting about 3.6 min/day.
I'll be trailing the drift.
The updates will cross the hour point again sometime tomorrow if there is no reset.
So, that last update will be just past the EOD... it'll actually be the 00-hour for the next day.


----------



## Jacky_BEL (Feb 28, 2021)

What is the meaning of time-slot:16 and Next:02 ?


----------



## cine.chris (Feb 28, 2021)

16th-hour updated at  2 minutes
2 is the minutes from the time when python wrote the file.
And a likely guess that the next update will occur at 2 minutes, which is a safe guess for ~4 days.
If I don't get the tracker working it skips 15 minutes.

I need to finish the tracker as the updates will cross the hour soon!  (see timestamp log below)
Those BASH shell script if's and arithmetic take a lot of testing and validation as I've worked with them much.  
And I guess I'm a clumsy programmer.  
Getting the opportunity to test these outlying conditions is hard to find or waiting another 4-5 days in this case.

20210228-170001
Sun Feb 28 21:55:59 GMT 2021
TimeStamp Trigger: PROCESSING


----------



## Jacky_BEL (Mar 1, 2021)

Thank you for taking the time to explain.
It's been a long time since I did some programming. I think you are doing a great job here!


----------



## cine.chris (Mar 1, 2021)

@Jacky_BEL Thanks.
Also, Time-Slot, I wanted to separate the updates from a clock-based event as they are continuing to drift.
Just a mental cue to expect update events to be fluid.

@Jacky_BEL Looking at #'s.
I show a later total update from Folding, but the daily base has shifted & daily #'s are too low.
Need to find that error.
It was correct at hour zero.

I checked the numbers for three donors, by hand, based on the GMT 5:53 data file,  the EOD for St Louis CST GMT-6.
File Header: Sun Feb 28 05:53:19 GMT 2021
and my web#s were dead-on exact for daily and the current totals were a later update than showing on EOC,
I was glancing at the 24/7data on EOC, thinking they didn't match.
EOC is likely still on a 3Hr update cycle and would have to be compared after the 3 hr update which takes ~20min.

@XZero450  must've found some additional gear, I have my edit station running today and he still passed me.

Top300 


			FAH RANK Data
		

I need to change the colors and rank scaling for this gang & add team


----------



## Jacky_BEL (Mar 1, 2021)

Okay,
So 1hr. and 3hr. updates are in fact seperate updates then?


----------



## cine.chris (Mar 1, 2021)

Jacky_BEL said:


> Okay,
> So 1hr. and 3hr. updates are in fact seperate updates then?


Same data, just the time, between when the updates occur.
My understanding of updates...
Historically, it had been hourly, at the end of the hour.  
Apparently, during the Covid work surge, FAH went to a 3hr update cycle.
Then recently, it returned to an almost hourly update rate.

@Jacky_BEL 
The update tracker is working.
An amazing <5min from timestamp to page publish.
About 1-2 min is lost in the update actually being available for download, based on data I reviewed this morning.
Based on that, I added a 3min buffer to the tracker...
so, it can't get much faster.
It also transitioned the hour rollover without issue.
This scheduling is tight, I'm expecting some hiccups.


----------



## cine.chris (Mar 5, 2021)

20210304-220001
sleeping 16 minutes
Fri Mar 05 03:13:51 GMT 2021  <== time when this line is written, 1st of 2.5million records that gets compressed, put online
TimeStamp Trigger: PROCESSING
---- 221747 ----    <========== time after the last page was transferred to web server (Dreamhost, Virginia, USA)
An impressive sub 4min update time.  Timestamp to Page Publish on the website.
Update File TimeStamp: 13:51
Page Publish completion: 17:47


----------



## cine.chris (Mar 8, 2021)

The FAH STATS have stalled... again.
Numbers do not reset as there wasn't an EOD (end-of-day) points update.
========================
STATS feed has resumed...

Also, missing data from two servers has been added to the database, so you might get an extra points bump too.
This is also was the reason the updates were stalled.


----------



## Jacky_BEL (Mar 9, 2021)

The numbers are showing in the EOC , DC and your stats, but the Donor stats Folding@home stats report and Folding@home Recent CPUs  are no longer updated.
They are still stuck at 2021-03-06 21:35:45 for me.


----------



## cine.chris (Mar 9, 2021)

They're working on replacing the old stat pages.
I only know what I've seen mentioned on Discord or in the Forums.
I got the BASH date code working yesterday, turns out it's simple & powerful.  I'd never had to do much shell scripting in the past.
So... stall detection, update timing, will be much easier, more reliable.
@Jacky_BEL
Found a link for you:






						Folding@home | Statistics
					

Folding@home Statistics




					statsbeta.foldingathome.org
				




There is an OS tab there...


----------



## Jacky_BEL (Mar 9, 2021)

I used the Folding@home Recent CPUs page for checking in on the systems running , while being away from home.
They should all report last WU returned within a certain timeperiod depending on the GPU and WU.
Most will finish a WU within 3 hours, my slowest may take up to 4 hours, depending on the WU given.
I found this method easy and good enough, no hassles installing or configuring.

I can't find a replacement for this at the link you found.

This was an app from the Folding@home web apps page.
If I try to get info for a newly finished WU status , with the PRCG details that I retrieve from the log file, I get a Not Found. error
I hope these apps will also get reworked / replaced.


----------



## cine.chris (Mar 9, 2021)

Currently, the update times have stabilized.  (see timestamps below)
This is timestamp from the previous run ~1hr earlier.
Which makes things simpler.
The page updates are running 1:30's, executing on the hour.
20210309-110001
diff minutes: 67
passthru mode: 55
Tue Mar 09 15:52:28 GMT 2021
TimeStamp Trigger: PROCESSING
---- 110131 ----
20210309-120001
diff minutes: 67
passthru mode: 55
Tue Mar 09 16:52:29 GMT 2021
TimeStamp Trigger: PROCESSING
---- 120134 ----
20210309-130001
diff minutes: 67
passthru mode: 55
Tue Mar 09 17:52:29 GMT 2021
TimeStamp Trigger: PROCESSING
---- 130136 ----
20210309-140001
diff minutes: 67
passthru mode: 55
Tue Mar 09 18:52:29 GMT 2021
TimeStamp Trigger: PROCESSING
---- 140138 ----


----------



## Jacky_BEL (Mar 10, 2021)

Jacky_BEL said:


> The numbers are showing in the EOC , DC and your stats, but the Donor stats Folding@home stats report and Folding@home Recent CPUs  are no longer updated.
> They are still stuck at 2021-03-06 21:35:45 for me.


F@h Stats report and Recent CPUs pages are working again. 
Good job from the F@h-team


----------



## cine.chris (Mar 11, 2021)

The FAH updates resumed their drift:
20210310-180001
diff minutes: 63
runat-sleep: 0 0
Wed Mar 10 22:56:18 GMT 2021
TimeStamp Trigger: PROCESSING
---- 180145 ----
Moving just shy of 4 minutes from times posted above.
The latest update to the tracker is much "smarter", actually tracking synced time across timezones.  
It's now reached the 3 minute buffer allowed for the update process to complete and will begin to add sleep minutes to allow the update to complete.


----------



## cine.chris (Mar 12, 2021)

New, "softer" colors (more pastel-like), easier to read.
Refactored code, put colors into an array. Simplified color management.
Depending on your browser cache, it might not show immediately.
The updates continue to drift, but the rate is more erratic.
So far... I've not seen the tracker miss an update.  
Stalls or erratic update server behavior...
Should that occur, the current code is adaptive; it should be smart enough to detect a shift and reset.
Checking for valid updates on the hour.


----------



## BWG (Mar 25, 2021)

Nice work man! Looks great.


----------

