The Linux NTFS filesystem driver

Overview

Linux-NTFS comes with a number of user-space programs known as ntfsprogs.These include mkntfs, a full-featured ntfs filesystem format utility,ntfsundelete used for recovering files that were unintentionally deletedfrom an NTFS volume and ntfsresize which is used to resize an NTFS partition.See the web site for more information.

To mount an NTFS 1.2/3.x (Windows NT4/2000/XP/2003) volume, use the filesystem type ‘ntfs’. The driver currently supports read-only mode (with nofault-tolerance, encryption or journalling) and very limited, but safe, writesupport.

For fault tolerance and raid support (i.e. volume and stripe sets), you canuse the kernel’s Software RAID / MD driver. See section “Using Software RAIDwith NTFS” for details.

Web site

There is plenty of additional information on the linux-ntfs web siteathttp://www.linux-ntfs.org/

The web site has a lot of additional information, such as a comprehensiveFAQ, documentation on the NTFS on-disk format, information on the Linux-NTFSuserspace utilities, etc.

Features

  • This is a complete rewrite of the NTFS driver that used to be in the 2.4 andearlier kernels. This new driver implements NTFS read support and isfunctionally equivalent to the old ntfs driver and it also implements limitedwrite support. The biggest limitation at present is that files/directoriescannot be created or deleted. See below for the list of write features thatare so far supported. Another limitation is that writing to compressed filesis not implemented at all. Also, neither read nor write access to encryptedfiles is so far implemented.

  • The new driver has full support for sparse files on NTFS 3.x volumes whichthe old driver isn’t happy with.

  • The new driver supports execution of binaries due to mmap() now beingsupported.

  • The new driver supports loopback mounting of files on NTFS which is used bysome Linux distributions to enable the user to run Linux from an NTFSpartition by creating a large file while in Windows and then loopbackmounting the file while in Linux and creating a Linux filesystem on it thatis used to install Linux on it.

  • A comparison of the two drivers using:

    time find . -type f -exec md5sum "{}" \;

    run three times in sequence with each driver (after a reboot) on a 1.4GiBNTFS partition, showed the new driver to be 20% faster in total time elapsed(from 9:43 minutes on average down to 7:53). The time spent in user spacewas unchanged but the time spent in the kernel was decreased by a factor of2.5 (from 85 CPU seconds down to 33).

  • The driver does not support short file names in general. For backwardscompatibility, we implement access to files using their short file names ifthey exist. The driver will not create short file names however, and arename will discard any existing short file name.

  • The new driver supports exporting of mounted NTFS volumes via NFS.

  • The new driver supports async io (aio).

  • The new driver supports fsync(2), fdatasync(2), and msync(2).

  • The new driver supports readv(2) and writev(2).

  • The new driver supports access time updates (including mtime and ctime).

  • The new driver supports truncate(2) and open(2) with O_TRUNC. But at presentonly very limited support for highly fragmented files, i.e. ones which havetheir data attribute split across multiple extents, is included. Anotherlimitation is that at present truncate(2) will never create sparse files,since to mark a file sparse we need to modify the directory entry for thefile and we do not implement directory modifications yet.

  • The new driver supports write(2) which can both overwrite existing data andextend the file size so that you can write beyond the existing data. Also,writing into sparse regions is supported and the holes are filled in withclusters. But at present only limited support for highly fragmented files,i.e. ones which have their data attribute split across multiple extents, isincluded. Another limitation is that write(2) will never create sparsefiles, since to mark a file sparse we need to modify the directory entry forthe file and we do not implement directory modifications yet.

Supported mount options

In addition to the generic mount options described by the manual page for themount command (man 8 mount, also see man 5 fstab), the NTFS driver supports thefollowing mount options:

iocharset=nameDeprecated option. Still supported but please usenls=name in the future. See description for nls=name.
nls=nameCharacter set to use when returning file names.Unlike VFAT, NTFS suppresses names that containunconvertible characters. Note that most charactersets contain insufficient characters to represent allpossible Unicode characters that can exist on NTFS.To be sure you are not missing any files, you areadvised to use nls=utf8 which is capable ofrepresenting all Unicode characters.
utf8=<bool>Option no longer supported. Currently mapped tonls=utf8 but please use nls=utf8 in the future andmake sure utf8 is compiled either as module or intothe kernel. See description for nls=name.
uid= 
gid= 
umask=Provide default owner, group, and access mode mask.These options work as documented in mount(8). Bydefault, the files/directories are owned by root andhe/she has read and write permissions, as well asbrowse permission for directories. No one else has anyaccess permissions. I.e. the mode on all files is bydefault rw——- and for directories rwx——, aconsequence of the default fmask=0177 and dmask=0077.Using a umask of zero will grant all permissions toeveryone, i.e. all files and directories will have moderwxrwxrwx.
fmask= 
dmask=Instead of specifying umask which applies both tofiles and directories, fmask applies only to files anddmask only to directories.
sloppy=<BOOL>If sloppy is specified, ignore unknown mount options.Otherwise the default behaviour is to abort mount ifany unknown options are found.
show_sys_files=<BOOL>If show_sys_files is specified, show the system filesin directory listings. Otherwise the default behaviouris to hide the system files.Note that even when show_sys_files is specified, “$MFT”will not be visible due to bugs/mis-features in glibc.Further, note that irrespective of show_sys_files, allfiles are accessible by name, i.e. you can always do“ls -l $UpCase” for example to specifically show thesystem file containing the Unicode upcase table.
case_sensitive=<BOOL>If case_sensitive is specified, treat all file names ascase sensitive and create file names in the POSIXnamespace. Otherwise the default behaviour is to treatfile names as case insensitive and to create file namesin the WIN32/LONG name space. Note, the Linux NTFSdriver will never create short file names and willremove them on rename/delete of the corresponding longfile name.Note that files remain accessible via their short filename, if it exists. If case_sensitive, you will needto provide the correct case of the short file name.
disable_sparse=<BOOL>If disable_sparse is specified, creation of sparseregions, i.e. holes, inside files is disabled for thevolume (for the duration of this mount only). Bydefault, creation of sparse regions is enabled, whichis consistent with the behaviour of traditional Unixfilesystems.
errors=opt

What to do when critical filesystem errors are found.Following values can be used for “opt”:

continueDEFAULT, try to clean-up as much aspossible, e.g. marking a corrupt inode asbad so it is no longer accessed, and thencontinue.
recoverAt present only supported is recovery ofthe boot sector from the backup copy.If read-only mount, the recovery is donein memory only and not written to disk.

Note that the options are additive, i.e. specifying:

errors=continue,errors=recover

means the driver will attempt to recover and if thatfails it will clean-up as much as possible andcontinue.

mft_zone_multiplier=

Set the MFT zone multiplier for the volume (thissetting is not persistent across mounts and can bechanged from mount to mount but cannot be changed onremount). Values of 1 to 4 are allowed, 1 being thedefault. The MFT zone multiplier determines how muchspace is reserved for the MFT on the volume. If allother space is used up, then the MFT zone will beshrunk dynamically, so this has no impact on theamount of free space. However, it can have an impacton performance by affecting fragmentation of the MFT.In general use the default. If you have a lot of smallfiles then use a higher value. The values have thefollowing meaning:

ValueMFT zone size (% of volume size)
112.5%
225%
337.5%
450%

Note this option is irrelevant for read-only mounts.

Known bugs and (mis-)features

  • The link count on each directory inode entry is set to 1, due to Linux notsupporting directory hard links. This may well confuse some user spaceapplications, since the directory names will have the same inode numbers.This also speeds up ntfs_read_inode() immensely. And we haven’t found anyproblems with this approach so far. If you find a problem with this, pleaselet us know.

Please send bug reports/comments/feedback/abuse to the Linux-NTFS developmentlist at sourceforge:linux-ntfs-dev@lists.sourceforge.net

Using NTFS volume and stripe sets

For support of volume and stripe sets, you can either use the kernel’sDevice-Mapper driver or the kernel’s Software RAID / MD driver. The former isthe recommended one to use for linear raid. But the latter is required forraid level 5. For striping and mirroring, either driver should work fine.

The Device-Mapper driver

You will need to create a table of the components of the volume/stripe set andhow they fit together and load this into the kernel using the dmsetup utility(see man 8 dmsetup).

Linear volume sets, i.e. linear raid, has been tested and works fine. Eventhough untested, there is no reason why stripe sets, i.e. raid level 0, andmirrors, i.e. raid level 1 should not work, too. Stripes with parity, i.e.raid level 5, unfortunately cannot work yet because the current version of theDevice-Mapper driver does not support raid level 5. You may be able to use theSoftware RAID / MD driver for raid level 5, see the next section for details.

To create the table describing your volume you will need to know each of itscomponents and their sizes in sectors, i.e. multiples of 512-byte blocks.

For NT4 fault tolerant volumes you can obtain the sizes using fdisk. So forexample if one of your partitions is /dev/hda2 you would do:

$ fdisk -ul /dev/hdaDisk /dev/hda: 81.9 GB, 81964302336 bytes255 heads, 63 sectors/track, 9964 cylinders, total 160086528 sectorsUnits = sectors of 1 * 512 = 512 bytes    Device Boot      Start         End      Blocks   Id  System    /dev/hda1   *          63     4209029     2104483+  83  Linux    /dev/hda2         4209030    37768814    16779892+  86  NTFS    /dev/hda3        37768815    46170809     4200997+  83  Linux

And you would know that /dev/hda2 has a size of 37768814 - 4209030 + 1 =33559785 sectors.

For Win2k and later dynamic disks, you can for example use the ldminfo utilitywhich is part of the Linux LDM tools (the latest version at the time ofwriting is linux-ldm-0.0.8.tar.bz2). You can download it from:

Simply extract the downloaded archive (tar xvjf linux-ldm-0.0.8.tar.bz2), gointo it (cd linux-ldm-0.0.8) and change to the test directory (cd test). Youwill find the precompiled (i386) ldminfo utility there. NOTE: You will not beable to compile this yourself easily so use the binary version!

Then you would use ldminfo in dump mode to obtain the necessary information:

$ ./ldminfo --dump /dev/hda

This would dump the LDM database found on /dev/hda which describes all of yourdynamic disks and all the volumes on them. At the bottom you will see theVOLUME DEFINITIONS section which is all you really need. You may need to lookfurther above to determine which of the disks in the volume definitions iswhich device in Linux. Hint: Run ldminfo on each of your dynamic disks andlook at the Disk Id close to the top of the output for each (the PRIVATE HEADERsection). You can then find these Disk Ids in the VBLK DATABASE section in the<Disk> components where you will get the LDM Name for the disk that is found inthe VOLUME DEFINITIONS section.

Note you will also need to enable the LDM driver in the Linux kernel. If yourdistribution did not enable it, you will need to recompile the kernel with itenabled. This will create the LDM partitions on each device at boot time. Youwould then use those devices (for /dev/hda they would be /dev/hda1, 2, 3, etc)in the Device-Mapper table.

You can also bypass using the LDM driver by using the main device (e.g./dev/hda) and then using the offsets of the LDM partitions into this device asthe “Start sector of device” when creating the table. Once again ldminfo wouldgive you the correct information to do this.

Assuming you know all your devices and their sizes things are easy.

For a linear raid the table would look like this (note all values are in512-byte sectors):

# Offset into       Size of this    Raid type       Device          Start sector# volume    device                                          of device0           1028161         linear          /dev/hda1       01028161             3903762         linear          /dev/hdb2       04931923             2103211         linear          /dev/hdc1       0

For a striped volume, i.e. raid level 0, you will need to know the chunk sizeyou used when creating the volume. Windows uses 64kiB as the default, so itwill probably be this unless you changes the defaults when creating the array.

For a raid level 0 the table would look like this (note all values are in512-byte sectors):

# Offset   Size         Raid     Number   Chunk  1st        Start   2nd       Start# into     of the   type     of           size   Device     in      Device    in# volume   volume        stripes                    device            device0      2056320  striped  2        128    /dev/hda1  0       /dev/hdb1 0

If there are more than two devices, just add each of them to the end of theline.

Finally, for a mirrored volume, i.e. raid level 1, the table would look likethis (note all values are in 512-byte sectors):

# Ofs Size   Raid   Log  Number Region Should Number Source  Start Target Start# in  of the type   type of log size   sync?  of     Device  in    Device in# vol volume                 params              mirrors         Device       Device0    2056320 mirror core 2  16     nosync 2    /dev/hda1 0   /dev/hdb1 0

If you are mirroring to multiple devices you can specify further targets at theend of the line.

Note the “Should sync?” parameter “nosync” means that the two mirrors arealready in sync which will be the case on a clean shutdown of Windows. If themirrors are not clean, you can specify the “sync” option instead of “nosync”and the Device-Mapper driver will then copy the entirety of the “Source Device”to the “Target Device” or if you specified multiple target devices to all ofthem.

Once you have your table, save it in a file somewhere (e.g. /etc/ntfsvolume1),and hand it over to dmsetup to work with, like so:

$ dmsetup create myvolume1 /etc/ntfsvolume1

You can obviously replace “myvolume1” with whatever name you like.

If it all worked, you will now have the device /dev/device-mapper/myvolume1which you can then just use as an argument to the mount command as usual tomount the ntfs volume. For example:

$ mount -t ntfs -o ro /dev/device-mapper/myvolume1 /mnt/myvol1

(You need to create the directory /mnt/myvol1 first and of course you can useanything you like instead of /mnt/myvol1 as long as it is an existingdirectory.)

It is advisable to do the mount read-only to see if the volume has been setupcorrectly to avoid the possibility of causing damage to the data on the ntfsvolume.

The Software RAID / MD driver

An alternative to using the Device-Mapper driver is to use the kernel’sSoftware RAID / MD driver. For which you need to set up your /etc/raidtabappropriately (see man 5 raidtab).

Linear volume sets, i.e. linear raid, as well as stripe sets, i.e. raid level0, have been tested and work fine (though see section “Limitations when usingthe MD driver with NTFS volumes” especially if you want to use linear raid).Even though untested, there is no reason why mirrors, i.e. raid level 1, andstripes with parity, i.e. raid level 5, should not work, too.

You have to use the “persistent-superblock 0” option for each raid-disk in theNTFS volume/stripe you are configuring in /etc/raidtab as the persistentsuperblock used by the MD driver would damage the NTFS volume.

Windows by default uses a stripe chunk size of 64k, so you probably want the“chunk-size 64k” option for each raid-disk, too.

For example, if you have a stripe set consisting of two partitions /dev/hda5and /dev/hdb1 your /etc/raidtab would look like this:

raiddev /dev/md0        raid-level  0        nr-raid-disks       2        nr-spare-disks      0        persistent-superblock       0        chunk-size  64k        device              /dev/hda5        raid-disk   0        device              /dev/hdb1        raid-disk   1

For linear raid, just change the raid-level above to “raid-level linear”, formirrors, change it to “raid-level 1”, and for stripe sets with parity, changeit to “raid-level 5”.

Note for stripe sets with parity you will also need to tell the MD driverwhich parity algorithm to use by specifying the option “parity-algorithmwhich”, where you need to replace “which” with the name of the algorithm touse (see man 5 raidtab for available algorithms) and you will have to try thedifferent available algorithms until you find one that works. Make sure youare working read-only when playing with this as you may damage your dataotherwise. If you find which algorithm works please let us know (email thelinux-ntfs developers listlinux-ntfs-dev@lists.sourceforge.net or drop in onIRC in channel #ntfs on the irc.freenode.net network) so we can update thisdocumentation.

Once the raidtab is setup, run for example raid0run -a to start all devices orraid0run /dev/md0 to start a particular md device, in this case /dev/md0.

Then just use the mount command as usual to mount the ntfs volume using forexample:

mount -t ntfs -o ro /dev/md0 /mnt/myntfsvolume

It is advisable to do the mount read-only to see if the md volume has beensetup correctly to avoid the possibility of causing damage to the data on thentfs volume.

Limitations when using the Software RAID / MD driver

Using the md driver will not work properly if any of your NTFS partitions havean odd number of sectors. This is especially important for linear raid as alldata after the first partition with an odd number of sectors will be offset byone or more sectors so if you mount such a partition with write support youwill cause massive damage to the data on the volume which will only becomeapparent when you try to use the volume again under Windows.

So when using linear raid, make sure that all your partitions have an evennumber of sectors BEFORE attempting to use it. You have been warned!

Even better is to simply use the Device-Mapper for linear raid and then you donot have this problem with odd numbers of sectors.