Size on Disk Excessively Different than Size

kilo · Apr 23, 2024

I recently got new drives; moving from two 3TB drives in RAID 0 to two 8TB drives in RAID 0. Both drives were veracrypted volumes, however, when I copied data to the new 8TB drives, the size on disk is different. One example below where the size is 2.68TB for the folder but 3.62TB size on disk. The only difference I know I made was the 3TB RAID disks were using a 5kB key file and the new 8TB RAID disks are using a >10kB key file. I've also used defraggler on the new drives, which ran 21 hours to defrag 4MB, as reported by defraggler.

I appreciate any advice or thoughts. This is kind of a new one for me and I'm stumped.

2024-04-22 20_32_06-Technical Properties.png

R-T-B · Apr 23, 2024

Lots of small, sub 4KB files not completely filling clusters, most likely.

Solaris17 · Apr 23, 2024

R-T-B said:
Lots of small, sub 4KB files not completely filling clusters, most likely.

im wiling to bet on this too. If you have lots of small files and you still have the data on another device, wipe and rebuild the array with a smaller stripe size. generally 64 should be ok for daily use. You can get away with 32 for raid 0. 16 is pushing it for IO on spinners. Anything higher than 64k should be for ISOs Videos or large files.

kilo · Apr 23, 2024

Both volumes were "default" (4kB?) when formatted. I will try with different sizes just in case though and report back in a few days.

thesmokingman · Apr 23, 2024

Take all those tiny files and zip them.

ty_ger · Apr 23, 2024

264,000 files in 26,000 folders?!

I'm not surprised. Could be far worse. There's a lot of potential for unfilled clusters.

Solaris17 · Apr 23, 2024

kilo said:
Both volumes were "default" (4kB?) when formatted. I will try with different sizes just in case though and report back in a few days.

Thats the file system, NTFS is default at 4kb (this is fine) the RAID stripe size is not the same as cluster size.

kilo · Apr 23, 2024

ty_ger said:
264,000 files in 26,000 folders?!

I'm not surprised. Could be far worse. There's a lot of potential for unfilled clusters.

That's os iso's, github code, software installation files, lightweight software packages, knowledge base articles saved locally. It's not very much for 15 years of information.

Solaris17 said:
Thats the file system, NTFS is default at 4kb (this is fine) the RAID stripe size is not the same as cluster size.

Of course I would forget about raid stripe sizes...

I haven't done it in 6 years. Thank you. I'll verify that too.

kilo · Apr 26, 2024

Checked RAID stripe size and it was 64kb, formatted as ntfs 4kb, and restored some data. Looks like it's normal sizes:

Size: 2.68 TB (2,947,426,839,102)
Size on disk: 2.68 TB (2,947,816,177,664)

Reformatted again with a new veracrypt key which will take another 13 hours (whole disk encryption). But it's looking like I tried to be a smarty pants with whatever veracrypt settings I used previously and misconfigured.

natr0n · Apr 26, 2024

kilo is dealing with byte discrepancies

kilo · Monday at 3:06 AM

The issue turned out to be the the default file system option on Vera Crypt. Selecting NTFS instead of exFAT fixed the issue.

System Name	Regular PC \| Server HP Z440
Processor	9700k \| E5-2698v3
Motherboard	Gigabyte Z390 Gaming X-CF \| Stock mobo
Cooling	Scythe Mugen 5 rev. B \| Stock HS
Memory	32 GB (8x4) \| 112 GB (8x2 + 16x6)
Video Card(s)	RTX 2070 Super \| K4000
Storage	970 EVO+ 1TB \| 860 1TB x2
Display(s)	XV340CK x2, 1080p x2
Power Supply	Corsair RM750x \| Stock
Software	Windows \| Proxmox 7

System Name	Pioneer
Processor	Ryzen R9 7950X
Motherboard	GIGABYTE Aorus Elite X670 AX
Cooling	Noctua NH-D15 + A whole lotta Sunon and Corsair Maglev blower fans...
Memory	64GB (4x 16GB) G.Skill Flare X5 @ DDR5-6000 CL30
Video Card(s)	XFX RX 7900 XTX Speedster Merc 310
Storage	2x Crucial P5 Plus 2TB PCIe 4.0 NVMe SSDs
Display(s)	55" LG 55" B9 OLED 4K Display
Case	Thermaltake Core X31
Audio Device(s)	TOSLINK->Schiit Modi MB->Asgard 2 DAC Amp->AKG Pro K712 Headphones or HDMI->B9 OLED
Power Supply	FSP Hydro Ti Pro 850W
Mouse	Logitech G305 Lightspeed Wireless
Keyboard	WASD Code v3 with Cherry Green keyswitches + PBT DS keycaps
Software	Gentoo Linux x64

System Name	Rocinante
Processor	I9 14900KS
Motherboard	EVGA z690 Dark KINGPIN (modded BIOS)
Cooling	EK-AIO Elite 360 D-RGB
Memory	64GB Gskill Trident Z5 DDR5 6000 @6400
Video Card(s)	MSI SUPRIM Liquid X 4090
Storage	1x 500GB 980 Pro \| 1x 1TB 980 Pro \| 1x 8TB Corsair MP400
Display(s)	Philips Evnia 49M2C8900
Case	Lian Li o11 Evo Dynamic White
Audio Device(s)	Moondrop S8's on Schiit Hel 2e
Power Supply	Bequiet! Power Pro 12 1500w
Mouse	Lamzu Atlantis mini (White)
Keyboard	Monsgeek M3 Lavender, Akko Crystal Blues
VR HMD	Quest 3
Software	Windows 11
Benchmark Scores	I dont have time for that.

System Name	Regular PC \| Server HP Z440
Processor	9700k \| E5-2698v3
Motherboard	Gigabyte Z390 Gaming X-CF \| Stock mobo
Cooling	Scythe Mugen 5 rev. B \| Stock HS
Memory	32 GB (8x4) \| 112 GB (8x2 + 16x6)
Video Card(s)	RTX 2070 Super \| K4000
Storage	970 EVO+ 1TB \| 860 1TB x2
Display(s)	XV340CK x2, 1080p x2
Power Supply	Corsair RM750x \| Stock
Software	Windows \| Proxmox 7

Processor	AMD 5900x
Motherboard	Asus x570 Strix-E
Cooling	Hardware Labs
Memory	G.Skill 4000c17 2x16gb
Video Card(s)	RTX 3090
Storage	Sabrent
Display(s)	Samsung G9
Case	Phanteks 719
Audio Device(s)	Fiio K5 Pro
Power Supply	EVGA 1000 P2
Mouse	Logitech G600
Keyboard	Corsair K95

Size on Disk Excessively Different than Size

kilo

R-T-B

Solaris17

Super Dainty Moderator

kilo

thesmokingman

ty_ger

Solaris17

Super Dainty Moderator

kilo

kilo

natr0n

kilo

System Name	Z77 Rev. 1
Processor	Intel Core i7 3770K
Motherboard	ASRock Z77 Extreme4
Cooling	Water Cooling
Memory	2x G.Skill F3-2400C10D-16GTX
Video Card(s)	EVGA GTX 1080
Storage	Samsung 850 Pro
Display(s)	Samsung 28" UE590 UHD
Case	Silverstone TJ07
Audio Device(s)	Onboard
Power Supply	Seasonic PRIME 600W Titanium
Mouse	EVGA TORQ X10
Keyboard	Leopold Tenkeyless
Software	Windows 10 Pro 64-bit
Benchmark Scores	3DMark Time Spy: 7695

System Name	natr0n-PC
Processor	Ryzen 5950x/5600x
Motherboard	B450 AORUS M
Cooling	EK AIO 360 - 6 fan action
Memory	Patriot - Viper Steel DDR4 (B-Die)(4x8GB)
Video Card(s)	EVGA 3070ti FTW
Storage	Various
Display(s)	PIXIO IPS 240Hz 1080P
Case	Thermaltake Level 20 VT
Audio Device(s)	LOXJIE D10 + Kinter Amp + 6 Bookshelf Speakers Sony+JVC+Sony
Power Supply	Super Flower Leadex III ARGB 80+ Gold 650W
Software	XP/7/8.1/10
Benchmark Scores	http://valid.x86.fr/79kuh6