Repository files navigation

Install from PyPI usingpip install FATtools [3] (easier) or downloading the source code (or the released packages) from here.

Born to re-sort directory entries in a FAT32 root table to cope with some hardware MP3 players' limits, it now provides full read/write support in Python 3 (both 32- and 64-bit) for FAT12/16/32 and exFAT filesystems, for hacking and recovering purposes.

Moreover:

• it is able to open disk partitioned with both MBR or GPT and to help in partitioning (universal MBR boot code included)
• it can transparently create, read and write Dynamic and Differencing VHD, VHDX[1], VDI and VMDK disk images
• it can convert disk images between different virtual formats and clone disks
• it can handle RAW disk images and BytesIO "RamDisk" images, also.
• it supports 4K sectors
• it can handle large clusters (128K, 256K) with FAT formats[2]
• it is able to merge Differencing VHDs

Following features are implemented (mostly in Python, with a few ctypes calls to handle disks natively; compatibility with Linux and macOS is not regularly tested):

• sector aligned read/writes with both file images and real disks
• sector based caching mechanism (for both reading and writing) to speed-up FAT and directory table operations
• run length encoded map (with tuples and dictionaries) of free clusters, free directory slots, allocated cluster chains
• transparent reading and writing of FAT12/16/32 and exFAT filesystems with FS boot-sector auto recognizer
• MBR and GPT partitions handling
• Long File Name and Unicode support
• tools to open, create, rename, list and delete files and directories, and to partition disks
• facilities to sort, clean and shrink directory tables and to wipe (zero) free space
• file fragmentation calculator
• mkfat tool to properly (partition and) apply a FAT12/16/32 or exFAT filesystem to a block device (file or disk) and let CHKDSK be happy with it (included exFAT compressed Up-Case table generator)

Obviously, since a filesystem is an extremely complex and delicate matter, and big bugs may lay around, you'll USE IT TOTALLY AT YOUR OWN RISK!
But it seems quite stable and useable, now.

The most fragile area (and, thus, subject to bugs) was the caching mechanism, that operates in different ways:

• intercepting small I/O (<= 512 bytes), which is cached in a small circular buffer. Bigger I/O bypasses the cache; when the cache is full, all dirty sectors are committed to disk and the cache buffer is zeroed. Sectors and buffers are paired with Python builtin dictionaries: this permits a good (from a Pythonic perspective) I/O speed during FAT and directory tables access;
• maintaining a dictionary of pre-decoded FAT indexes, to improve the speed of repetitive access to cluster chains;
• maintaining a dictionary of short and long names (paired with their respective directory slots) for each directory table, to speed up searches and updates in directory tables;
• maintaining a RLE map of free clusters, free directory slots and allocated cluster chains, to dramatically improve speed of allocation and file access.

Actually, the I/O speed is closer to system's one.

Code is GPLed (look at GPL.TXT).

[1] VHDX Log support is actually limited to replaying capability.

[2] Actually, to say, one can partition with GPT an 8 TB VHDX with 4K sectors and format with FAT32 and happily use it under Windows 11. However, Windows 11 CHKDSK reports no more than 4 TBbytes (while it countsclusters correctly). Also, FORMAT itself can't apply such legitimate FAT32 format to an 8 TB disk.

[3] In Linux and macOS avenv is the preferred way.

The package installs afattools script, you can use this to perform simple command line operations.

NOTE: administrator (root, superuser) rights arealways required to access raw disk devices!

• to create a dynamic 8TB VHDX disk image with a single GPT partition and format it with exFAT:

fattools mkvdisk -s 8T --large-sectors image.vhdxfattools mkfat -t exfat -p gpt image.vhdx

• to create a differencing VDI disk image:

fattools mkvdisk -b image.vdi delta.vdi

• to wipe free space in an (ex)FAT formatted disk, zeroing all free clusters:

fattools wipe image.vhd

• to convert a RAW disk image into a Dynamic VHD (so implicitly virtualizing zeroed data blocks):

fattools imgclone image.raw image.vhd

Please note that resulting image size can get reduced if: 1) volume(s) is/are defragmented; 2) directory tables are cleaned and shrunk; 3) the free space has been wiped (zeroed) before.

• to capture a physical drive (disk block device) to a Dynamic VHD:
  fattools imgclone \\.\PhysicalDrive2 image.vhd (Windows)
  fattools imgclone /dev/sdb image.vhd (Linux)
  fattools imgclone /dev/disk0 image.vhd (macOS X, disk has first to be dismounted withdiskutil unmount)

• to list contents in a disk or disk image, copy items to/from it, display and erase them:

fattools ls \\.\PhysicalDrive2/py*fattools ls image1.vhd/py* image2.vdi/py*fattools cp C:\Python39\Lib\site-packages image.vhd/Python39/Libfattools cp image.vhd/Python39 C:\ProgramDatafattools cat image.vhd/readme.txtfattools rm image.vhd/Python39

• to open an existing plain or VHD disk image, or real disk:

# -*- coding: cp1252 -*-from FATtools.Volume import *disk = vopen('MyDiskImage.img', 'r+b', 'disk')

• to make a single GPT partition from all disk space:

from FATtools import partutilsgpt = partutils.partition(disk)

• to format such partition with the exFAT file system:

from FATtools import mkfat, Volumepart = Volume.vopen('MyDiskImage.img', 'r+b', 'partition0')mkfat.exfat_mkfs(part, part.size)

• to order items inside directory tables easily, with GUI and drag support (please note: in Linux, this requires Pythontkinter module previously installed bysudo apt-get install python3-tk):

fattools reordergui

• to order root directory table in USB drive X (scripting):

# -*- coding: cp1252 -*-from FATtools.Volume import *# Assuming we have DirA, DirB, DirC in this disk order into X:root = vopen('X:', 'r+b')new_order = '''DirBDirCDirA'''root._sortby.fix = new_order.split('\n') # uses built-in directory sort algorithmroot.sort(root._sortby) # user-defined order, in _sortby.fix listroot.sort() # default ordering (alphabetical)

• mixed access with Python and FATtools from the same script:

# -*- coding: cp1252 -*-from FATtools.Volume import vopen, vclosefrom FATtools.mkfat import exfat_mkfsfrom os.path import joinimport osreal_fat_fs = 'F:' # replace with mount point of your file system# Open and format with FATtoolsfs = vopen(real_fat_fs, 'r+b',what='disk')exfat_mkfs(fs, fs.size)vclose(fs)# Write some files with Python and list themT = ('c','a','b','d')for t in T:   open(join(real_fat_fs, t+'.txt'), 'w').write('This is a sample "%s.txt" file.'%t)print(os.listdir(real_fat_fs+'/'))# Open again, and sort root with FATtoolsfs = vopen(real_fat_fs, 'r+b')fs.sort()vclose(fs)# Check new table order with Pythonprint(os.listdir(real_fat_fs+'/'))

• (almost) same as above:

# -*- coding: cp1252 -*-from FATtools.Volume import vopen, vclosefrom FATtools.mkfat import exfat_mkfsfrom FATtools.partutils import partition# Open & create GPT partitiono = vopen('\\\\.\\PhysicalDrive1', 'r+b',what='disk')print('Partitioning...')partition(o, 'mbr')vclose(o)# Reopen and format with EXFATo = vopen('\\\\.\\PhysicalDrive1', 'r+b',what='partition0')print('Formatting...')exfat_mkfs(o, o.size)vclose(o) # auto-close partition AND disk# Reopen FS and writeprint('Writing...')o = vopen('\\\\.\\PhysicalDrive1', 'r+b')# Write some files with FATtools and sort themT = ('c','a','b','d')for t in T:   f = o.create(t+'.txt')   f.write(b'This is a sample "%s.txt" file.'%bytes(t,'ascii'))   f.close()o.sort()vclose(o)

Please look inside 'samples' directory for more usage samples.