TPU's WCG/BOINC Team

thebluebumblebee · Mar 3, 2023

"a disc failure" Seriously?

Edit: Actually, it was their RAID card, not a single disc failure.

Norton · Mar 3, 2023

They posted an update on Twitter saying they should be back in operation today. Mobile atm so maybe another team member can grab a link or screenshot?

thebluebumblebee · Mar 3, 2023

https://twitter.com/i/web/status/1631745931048566784

It gets worse:

https://twitter.com/i/web/status/1631819263060201472

As @BUCK NASTY used to say, looks like a good time to clean your systems!

Norton · Mar 4, 2023

Guess I'll fire up the RTX 3070 and get a little extra folding going over the weekend....

Toothless · Mar 4, 2023

Ah so that's where the WCG work went. Was wondering why it got a little cold in the living room.

Side note, I know I'm late but I'm glad you're back @Norton

phill · Mar 4, 2023

thebluebumblebee said:
As @BUCK NASTY used to say, looks like a good time to clean your systems!

If that's the case, we'd be loving some pics of those freshly cleaned systems

(Please excuse the bad pun there....!)

stinger608 · Mar 4, 2023

thebluebumblebee said:
As @BUCK NASTY used to say, looks like a good time to clean your systems!

Speaking of, where has he been?????? Did something happen that I'm no aware of?

theonedub · Mar 4, 2023

The migration from IBM to the current team was rough. I'll have to get my Threadripper fired back up sometime, hasn't been on in many many months now. Longest I've not been crunching since 2009. Life getting busy.

freeagent · Mar 4, 2023

I ran out of work this morning. I only noticed because my fans were quiet.. I thought it crashed or something.. nope. I think I have it set to grab a weeks worth at a time, cant remember..

dont whant to set it"' · Mar 5, 2023

Might as well make the most of the downtime , unplanned as it was and power tune down some more, I shall.
The household where I live just got its gas and electricity rates doubled(~96%increase)as of 1st this month.

xtreemchaos · Mar 5, 2023

a million piece jigsaw falls it to place at least thats what we are waiting for eh

.

dont whant to set it"' · Mar 5, 2023

Actually ,I've scheduled no new tasks a week ago, all completed 3and a half days ago, results not uploaded.

windwhirl · Mar 6, 2023

New day, new problems, I guess...
So, does anyone have WUs stuck in "Sending" (or whatever the word is in the English BOINC Manager)? Because I have like two and a half pages of WUs waiting to be sent..

dont whant to set it"' · Mar 6, 2023

Yes , mine is also at results upload pending/project backoff when I last checked this morning .

80-watt Hamster · Mar 6, 2023

windwhirl said:
New day, new problems, I guess...
So, does anyone have WUs stuck in "Sending" (or whatever the word is in the English BOINC Manager)? Because I have like two and a half pages of WUs waiting to be sent..
View attachment 286633

Yeah, my uploads have been stuck for days.

Arjai · Mar 6, 2023

I am having the same thing happen. All my jobs are done!

thebluebumblebee · Mar 7, 2023

6 hours ago....

https://twitter.com/i/web/status/1632845779889971201

NastyHabits · Mar 7, 2023

I just read this today on the WCG Facebook page. It was posted six days ago.

As of 6am today, we have been investigating a disk failure that has prevented access to our network file system. Until it has been fixed, the BOINC & website/forums databases and science filesystems will be unavailable. We will post a follow-up when it has been resolved.
Update: Our RAID controller card failed, so SHARCNET provided us with a spare. We can use it to hopefully restore the configuration and access the storage room soon.

Update #2: The borrowed RAID card worked and the drive layout was recognized, so we have all data safe (there is also a tape backup, but accessing that would be slower). Data center managed a full boot and we expect we will resume operation later today.

Update #3: All disks have finished rebuilding and the MySQL and DB2 filesystems are mounted. We are currently checking the /science filesystem integrity.

Update #4: We have confirmed all the data is intact and have replaced the RAID controller, but we are still having some issues with getting the new hardware production ready. Unfortunately, data center staff will not be able to help us over the weekend.
Additionally, the deadline of all existing WUs that are partially done will be extended and accepted once the hardware change is done.

phill · Mar 8, 2023

I think this captures everything so far.....?

I think @thebluebumblebee might have grabbed the same so apologies if double posting in a way...

I think they need to pay someone else to do this, its been a bad transfer and all these problems... Wonder what the heck is going on!

Another update.....

I do wonder rather a lot if Krembil had the knowledge or backing for such a massive project and if whether or not, they have the correct backing for what they actually need to keep this going and working as it used to with IBM..
Not massively detailed either which is frustrating but.... I dunno... I'd rather not let everything I have go to waste if WCG don't get their acts together...

phill · Mar 9, 2023

Another update everyone!!

Some sort of progress I think!!

NastyHabits · Mar 9, 2023

Looks like everything that could go wrong did. Glad they are down to one last item, although repairing a raid drive isn't trivial. Let us hope. Also, during my lifetime of being a PC desktop and server tech, I've never witnessed a PCI bus failure.

stinger608 · Mar 10, 2023

I hope they get up and running soon, otherwise, I'm shutting down the two crunchers I have running. They have been out of work units for close to 2 weeks now.

Haven't uploaded a single finished work unit. So probably going to loose all of them. :mad:

freeagent · Mar 10, 2023

Super unreliable after their big move.

1freedude · Mar 10, 2023

stinger608 said:
I hope they get up and running soon, otherwise, I'm shutting down the two crunchers I have running. They have been out of work units for close to 2 weeks now.

Haven't uploaded a single finished work unit. So probably going to loose all of them.

Hold on buddy, you don't have to give up. They said all work will be extended. Give it a few more days at least.

NastyHabits · Mar 10, 2023

freeagent said:
Super unreliable after their big move.

@freeagent . I spent 14 years as a computer tech in a large firm. In all that time we never saw the kind of break-down in equipment Kremble has had. I suspect that either they were using really old used servers, or their cooling system is ineffective or both.

System Name	Main/DC
Processor	i7-3770K/i7-2600K
Motherboard	MSI Z77A-GD55/GA-P67A-UD4-B3
Cooling	Phanteks PH-TC14CS/H80
Memory	Crucial Ballistix Sport 16GB (2 x 8GB) LP /4GB Kingston DDR3 1600
Video Card(s)	Asus GTX 660 Ti/MSI HD7770
Storage	Crucial MX100 256GB/120GB Samsung 830 & Seagate 2TB(died)
Display(s)	Asus 24' LED/Samsung SyncMaster B1940
Case	P100/Antec P280 It's huge!
Audio Device(s)	on board
Power Supply	SeaSonic SS-660XP2/Seasonic SS-760XP2
Software	Win 7 Home Premiun 64 Bit

System Name	Main PC- Gamer- Main Cruncher/Folder and too many crunching/folding rigs
Processor	Ryzen 5900X- Ryzen 5950X- Ryzen 3950X and etc...
Motherboard	Asrock X570 Extreme4- MSI X570S Tomahawk MAX WiFi- MSI B450M Bazooka Max and etc...
Cooling	Noctua NH-U14S (dual fan)- EK 360 AIO with push/pull fans- Corsair H115i RGB Pro XT and etc...
Memory	2x16GB GSkill FlareX 3200/c14- 4x8GB Corsair Vengeance 3600/c16- 2x16GB Team 3600/c18 and etc..
Video Card(s)	MSI Gaming RX 6800- Asus RTX 3070 TUF OC- MSI Ventus GTX 1660Ti and etc...
Storage	Main PC (1TB WD SN850- 2TB PNY CS 3040- 2TB Seagate Firecuda) and etc...
Display(s)	Main PC (2x24" Dell UltraSharp U2414H)
Case	Phanteks P600s- Seasonic Q704- Fractal Meshify C and etc...
Audio Device(s)	Logitech Z625 THX 2.1 speakers
Power Supply	EVGA 750 G3- SeaSonic DGC 750- EVGA P2 850 and etc...
Mouse	G300s
Keyboard	Corsair K65
VR HMD	N/A
Software	Windows 10 Pro or Ubuntu
Benchmark Scores	Why sit on the Bench when you can get in the game and Crunch!!!

System Name	Main/DC
Processor	i7-3770K/i7-2600K
Motherboard	MSI Z77A-GD55/GA-P67A-UD4-B3
Cooling	Phanteks PH-TC14CS/H80
Memory	Crucial Ballistix Sport 16GB (2 x 8GB) LP /4GB Kingston DDR3 1600
Video Card(s)	Asus GTX 660 Ti/MSI HD7770
Storage	Crucial MX100 256GB/120GB Samsung 830 & Seagate 2TB(died)
Display(s)	Asus 24' LED/Samsung SyncMaster B1940
Case	P100/Antec P280 It's huge!
Audio Device(s)	on board
Power Supply	SeaSonic SS-660XP2/Seasonic SS-760XP2
Software	Win 7 Home Premiun 64 Bit

System Name	Main PC- Gamer- Main Cruncher/Folder and too many crunching/folding rigs
Processor	Ryzen 5900X- Ryzen 5950X- Ryzen 3950X and etc...
Motherboard	Asrock X570 Extreme4- MSI X570S Tomahawk MAX WiFi- MSI B450M Bazooka Max and etc...
Cooling	Noctua NH-U14S (dual fan)- EK 360 AIO with push/pull fans- Corsair H115i RGB Pro XT and etc...
Memory	2x16GB GSkill FlareX 3200/c14- 4x8GB Corsair Vengeance 3600/c16- 2x16GB Team 3600/c18 and etc..
Video Card(s)	MSI Gaming RX 6800- Asus RTX 3070 TUF OC- MSI Ventus GTX 1660Ti and etc...
Storage	Main PC (1TB WD SN850- 2TB PNY CS 3040- 2TB Seagate Firecuda) and etc...
Display(s)	Main PC (2x24" Dell UltraSharp U2414H)
Case	Phanteks P600s- Seasonic Q704- Fractal Meshify C and etc...
Audio Device(s)	Logitech Z625 THX 2.1 speakers
Power Supply	EVGA 750 G3- SeaSonic DGC 750- EVGA P2 850 and etc...
Mouse	G300s
Keyboard	Corsair K65
VR HMD	N/A
Software	Windows 10 Pro or Ubuntu
Benchmark Scores	Why sit on the Bench when you can get in the game and Crunch!!!

System Name	Veral
Processor	7800x3D
Motherboard	x670e Asus Crosshair Hero
Cooling	Corsair H150i RGB Elite
Memory	2x24 Klevv Cras V RGB
Video Card(s)	Powercolor 7900XTX Red Devil
Storage	Crucial P5 Plus 1TB, Samsung 980 1TB, Teamgroup MP34 4TB
Display(s)	Acer Nitro XZ342CK Pbmiiphx, 2x AOC 2425W, AOC I1601FWUX
Case	Fractal Design Meshify Lite 2
Audio Device(s)	Blue Yeti + SteelSeries Arctis 5 / Samsung HW-T550
Power Supply	Corsair HX850
Mouse	Corsair Nightsword
Keyboard	Corsair K55
VR HMD	HP Reverb G2
Software	Windows 11 Professional
Benchmark Scores	PEBCAK

TPU's WCG/BOINC Team

thebluebumblebee

Norton

Moderator - Returning from the Darkness

thebluebumblebee

Norton

Moderator - Returning from the Darkness

Toothless

Tech, Games, and TPU!

phill

Moderator

stinger608

Dedicated TPU Cruncher & Folder

theonedub

habe fidem

freeagent

Moderator

dont whant to set it"'

xtreemchaos

dont whant to set it"'

windwhirl

dont whant to set it"'

80-watt Hamster

Arjai

thebluebumblebee

NastyHabits

phill

Moderator

phill

Moderator

NastyHabits

stinger608

Dedicated TPU Cruncher & Folder

freeagent

Moderator

1freedude

NastyHabits

System Name	Not so complete or overkill - There are others!! Just no room to put! :D
Processor	Ryzen Threadripper 3970X
Motherboard	Asus Zenith 2 Extreme Alpha
Cooling	Lots!! Dual GTX 560 rads with D5 pumps for each rad. One rad for each component
Memory	Viper Steel 4 x 16GB DDR4 3600MHz not sure on the timings... Probably still at 2667!! :(
Video Card(s)	Asus Strix 3090 with front and rear active full cover water blocks
Storage	I'm bound to forget something here - 250GB OS, 2 x 1TB NVME, 2 x 1TB SSD, 4TB SSD, 2 x 8TB HD etc...
Display(s)	3 x Dell 27" S2721DGFA @ 7680 x 1440P @ 144Hz or 165Hz - working on it!!
Case	The big Thermaltake that looks like a Case Mods
Audio Device(s)	Onboard
Power Supply	EVGA 1600W T2
Mouse	Corsair thingy
Keyboard	Razer something or other....
VR HMD	No headset yet
Software	Windows 11 OS... Not a fan!!
Benchmark Scores	I've actually never benched it!! Too busy with WCG and FAH and not gaming! :( :( Not OC'd it!! :(

System Name	Dean Machine/2020 Ryzenfall
Processor	Intel 4790K/AMD Ryzen 5800X
Motherboard	MSI 1150 Gaming mATX/Asus ROG STRIX B550-F Gaming Wifi
Cooling	Cooler Master Hyper 212 LED/SilverStone AH240 AIO
Memory	16 gigs Crucial Ballistix Tactical Tracer/32 gigs G.Skill TridentZ NEO DDR4
Video Card(s)	Gigabyte 1660 Super/EVGA GTX 1080 FTW Hybrid Gaming
Storage	Crucial SSD 256 and 2TB spinner/Dual Samsung 980 Pro M2 NVME 4.0
Display(s)	Overlord 27" 2560 x 1440
Case	Corsair Air 540
Audio Device(s)	On board
Power Supply	Seasonic modular 850 watt Platinum/EVGA T2-850 Titanium
Software	Windows 10 Pro/Windows 10 Pro

System Name	Erakith \|\| AsRock NUCS
Processor	AMD ThreadRipper 2990WX ES (32c/64t 3.5ghz@1.02v) \|\| Intel i7 1360P Raptor Lake-P
Motherboard	MSI X399 MEG Creation \|\| AsRock OE
Cooling	EK Supremacy TR4 Full Nickel, D5 w/ EK Top, 2X EK-CoolStream XTX 360 & Photon 270 Res \|\| OE
Memory	32GB (4x8GB) Corsair Vengeance PRO RGB DDR4 3000\|\| 2x16GB Corsair DDR4
Video Card(s)	Vega 64 EK FC\|\| Intel Iris Xe
Storage	250gb WD Black NVMe & (2) 2TB Intel 660p NVMe \|\| 2TB Corsair MP600 Pro
Display(s)	34" Dell Ultrasharp U3415W
Case	CaseLabs Merlin SM8 \|\| Aluminum Unibody
Power Supply	Corsair HX1000
Mouse	Corsair Harpoon Wireless RGB
Keyboard	Corsair Strafe RGB w/ Cherry MX Silent
Software	Windows 10 Pro x64 \|\| WIndows 11 Pro x64
Benchmark Scores	WCG & F@H Only

Processor	AMD R7 5800X3D
Motherboard	Asus Crosshair VIII Dark Hero
Cooling	Thermalright Frozen Edge 360, 3x TL-B12 V2, 2x TL-B12 V1
Memory	2x8 G.Skill Trident Z Royal 3200C14, 2x8GB G.Skill Trident Z Black and White 3200 C14
Video Card(s)	Zotac 4070 Ti Trinity OC
Storage	WD SN850 1TB, SN850X 2TB, SN770 1TB
Display(s)	LG 50UP7100
Case	Fractal Torrent Compact
Audio Device(s)	JBL Bar 700
Power Supply	Seasonic Vertex GX-1000, Monster HDP1800
Mouse	Logitech G502 Hero
Keyboard	Logitech G213
VR HMD	Oculus 3
Software	Yes
Benchmark Scores	Yes

System Name	3 "rigs"-gaming/spare pc/cruncher
Processor	R7-5800X3D/i7-7700K/R9-7950X
Motherboard	Asus ROG Crosshair VI Extreme/Asus Ranger Z170/Asus ROG Crosshair X670E-GENE
Cooling	Bitspower monoblock ,custom open loop,both passive and active/air tower cooler/air tower cooler
Memory	32GB DDR4/32GB DDR4/64GB DDR5
Video Card(s)	Gigabyte RX6900XT Alphacooled/AMD RX5700XT 50th Aniv./SOC(onboard)
Storage	mix of sata ssds/m.2 ssds/mix of sata ssds+an m.2 ssd
Display(s)	Dell UltraSharp U2410 , HP 24x
Case	mb box/Silverstone Raven RV-05/CoolerMaster Q300L
Audio Device(s)	onboard/onboard/onboard
Power Supply	3 Seasonics, a DeltaElectronics, a FractalDesing
Mouse	various/various/various
Keyboard	various wired and wireless
VR HMD	-
Software	W10.someting or another,all 3

System Name	1.FortySe7en VR rig 2. intel teliscope rig 3.MSI GP72MVR Leopard Pro .E-52699, Xeon play thing
Processor	1.3900x @stock 2. i7 7700k @5. 3. i7 7700hq
Motherboard	1.aorus x570 ultra 2. z270 Maximus IX Hero,4 MR9A PRO ATX X99
Cooling	1.Hard tube loop, cpu and gpu 2. Hard loop cpu and gpu 4 360 AIO
Memory	1.Gskill neo @3600 32gb 2.hyperxfury 32gb @3000 3. 16gb hyperx @2400 4 64GB 2133 in quad channel
Video Card(s)	1.GIGABYTE RTX 3080 WaterForce WB 2. Aorus RTX2080 3. 1060 3gb. 4 Arc 770LE 16 gb
Storage	1 M.2 500gb , 2 3tb HDs 2. 256gb ssd, 3tbHD 3. 256 m.2. 1tb ssd 4. 2gb ssd
Display(s)	1.LG 50" UHD , 2 MSI Optix MAG342C UWHD. 3.17" 120 hz display 4. Acer Preditor 144hz 32inch.z
Case	1. Thermaltake P5 2. Thermaltake P3 4. some cheapo case that should not be named.
Audio Device(s)	1 Onboard 2 Onboard 3 Onboard 4. onboard.
Power Supply	1.seasonic gx 850w 2. seasonic gx 750w. 4 RM850w
Mouse	1 ROG Gladius 2 Corsair m65 pro
Keyboard	1. ROG Strix Flare 2. Corsair F75 RBG 3. steelseries RBG
VR HMD	rift and rift S and Quest 2.
Software	1. win11 pro 2. win11 pro 3, win11 home 4 win11 pro
Benchmark Scores	1.7821 cb20 ,cb15 3442 1c 204 cpu-z 1c 539 12c 8847 2. 1106 cb 3.cb 970

System Name	System V
Processor	AMD Ryzen 5 3600
Motherboard	Asus Prime X570-P
Cooling	Cooler Master Hyper 212 // a bunch of 120 mm Xigmatek 1500 RPM fans (2 ins, 3 outs)
Memory	2x8GB Ballistix Sport LT 3200 MHz (BLS8G4D32AESCK.M8FE) (CL16-18-18-36)
Video Card(s)	Gigabyte AORUS Radeon RX 580 8 GB
Storage	SHFS37A240G / DT01ACA200 / ST10000VN0008 / ST8000VN004 / SA400S37960G / SNV21000G / NM620 2TB
Display(s)	LG 22MP55 IPS Display
Case	NZXT Source 210
Audio Device(s)	Logitech G430 Headset
Power Supply	Corsair CX650M
Software	Whatever build of Windows 11 is being served in Canary channel at the time.
Benchmark Scores	Corona 1.3: 3120620 r/s Cinebench R20: 3355 FireStrike: 12490 TimeSpy: 4624

System Name	Office
Processor	Ryzen 5600G
Motherboard	ASUS B450M-A II
Cooling	be quiet! Shadow Rock LP
Memory	16GB Patriot Viper Steel DDR4-3200
Video Card(s)	Gigabyte RX 5600 XT
Storage	PNY CS1030 250GB, Crucial MX500 2TB
Display(s)	Dell S2719DGF
Case	Fractal Define 7 Compact
Power Supply	EVGA 550 G3
Mouse	Logitech M705 Marthon
Keyboard	Logitech G410
Software	Windows 10 Pro 22H2

System Name	Bay2- Lowerbay/ HP 3770/T3500-2+T3500-3+T3500-4/ Opti-Con/Orange/White/Grey
Processor	i3 2120's/ i7 3770/ x5670's/ i5 2400/Ryzen 2700/Ryzen 2700/R7 3700x
Motherboard	HP UltraSlim's/ HP mid size/ Dell T3500 workstation's/ Dell 390/B450 AorusM/B450 AorusM/B550 AorusM
Cooling	All stock coolers/Grey has an H-60
Memory	2GB/ 4GB/ 12 GB 3 chan/ 4GB sammy/T-Force 16GB 3200/XPG 16GB 3000/Ballistic 3600 16GB
Video Card(s)	HD2000's/ HD 2000/ 1 MSI GT710,2x MSI R7 240's/ HD4000/ Red Dragon 580/Sapphire 580/Sapphire 580
Storage	?HDD's/ 500 GB-er's/ 500 GB/2.5 Samsung 500GB HDD+WD Black 1TB/ WD Black 500GB M.2/Corsair MP600 M.2
Display(s)	1920x1080/ ViewSonic VX24568 between the rest/1080p TV-Grey
Case	HP 8200 UltraSlim's/ HP 8200 mid tower/Dell T3500's/ Dell 390/SilverStone Kublai KL06/NZXT H510 W x2
Audio Device(s)	Sonic Master/ onboard's/ Beeper's!
Power Supply	19.5 volt bricks/ Dell PSU/ 525W sumptin/ same/Seasonic 750 80+Gold/EVGA 500 80+/Antec 650 80+Gold
Mouse	cheap GigaWire930, CMStorm Havoc + Logitech M510 wireless/iGear usb x2/MX 900 wireless kit 4 Grey
Keyboard	Dynex, 2 no name, SYX and a Logitech. All full sized and USB. MX900 kit for Grey
Software	Mint 18 Sylvia/ Opti-Con Mint KDE/ T3500's on Kubuntu/HP 3770 is Win 10/Win 10 Pro/Win 10 Pro/Win10
Benchmark Scores	World Community Grid is my benchmark!!

System Name	Rex
Processor	Ryzen 2600
Motherboard	GigaByte B450M DS3H
Cooling	Wraith
Memory	16 GB Corsair Vengence 3000
Video Card(s)	Evga GTX 1660 SC
Storage	XPG M.2, Samsung 860 SSD, WD Black 1 TB
Display(s)	ASUS VG248
Case	Corsair mATX cube. (Alas they don't make it anymore)
Power Supply	Corsair HX 600 Watt
Mouse	Microsoft
Keyboard	Logitech
Software	Windows 10 64 ibit

Processor	Xeon x5650
Motherboard	SABERTOOTH X58
Cooling	Fans
Memory	24 GB Kingston HyperX 1600
Video Card(s)	GTX 1060 3GB
Storage	small ssd
Display(s)	Dell 2001F, BenQ short throw
Case	Lian Li
Audio Device(s)	onboard
Power Supply	X750
Software	Mint 19.3, Win 10
Benchmark Scores	not so fast...