Feb 012012
 

Disks are becoming cheaper and so it’s becoming common to have on our desk an external (usually USB) hard drive with a size of 1TB or more.
Now, once that we bring at home our new gadget the first question for us is: “which Filesystem should i use with this big disk ?”

There are a lot of options and the main point is once again a simple question: do you want to use this disk with several different operating systems ?
Perhaps you have a computer with a dual-boot with a Windows operating system , or perhaps you plan to use that external hard drive also with your Macbook…or connect it to a TV.

Let’s take a look at some options you have to suit your needs.

Multi-Partition solution

A possible solution is to split your hard drive in multiple partitions and dedicate each one to an Operating system, so for example you could have 25% of your disk with a Fat32, 25% with NTFS and 50% with Ext4. Personally i don’t like too much this solution, sure you can use a native file system in every operating system, but you also have all the possible problems in mounting a different FS, and I’m sure that I happen to have the right documents in the wrong partition.

So, for me the best solution is to have 1 partition and 1 big filesystem.

The most common File systems

The most common choices when you think at a USB hard disk are FAT32, NTFS, ext3 and ext4 (well this one is a less common than the others i suppose),, so as first thing let’s take a look at the limits of these filesystems:

File size limit:
FAT32 : 4GiB
NTFS : 16 EB
ext3 : 2 TiB
ext4 : 16 TiB

Partition size limit:
FAT32 : 2TiB
NTFS : 2TiB
ext3 : 32TiB
ext4 : 1 EiB

Taking a quick look at these information i’d discard FAT32, is becoming common to have big files, such as the backup of a Bluray or a DVD and the technology behind this FS is really old, the good thing of this FS is that is well supported by all operating systems.
Note: FAT32 theoretically has a volume limit of 2TB, but Windows won’t let you format a partition larger than ~32GB as FAT32 so you’d have to format it in Linux.

The other FS that at the moment i discard is ext4, This FS is good if you use only Linux, but there is no support on WIndows for ext4.

NTFS

If your primary operating system is windows this is probably the best choice, This FS is compatible with OSX and there is a good read/write support on Linux with the NTFS-3G driver. NTFS-3G supports all operations for writing files: files of any size can be created, modified, renamed, moved, or deleted on NTFS partitions. Transparent compression is supported, but there is no support for encryption.Support to modify access control lists and permissions is available. NTFS partitions are mounted using the Filesystem in Userspace (FUSE) interface.


Ext3

If your primary operating system is Linux ext3 could be your filesystem. It’s reliable and fast enough for a general use and some operations are supported on Windows, thanks to ext2fsd an open source ext3/4 file system driver for Windows (2K/XP/VISTA/WIN7).

Features Ext2Fsd supports:

  • ext2/ext3 volume reading & writing
  • ext3 journal replay when mounting
  • various codepage: utf8, cp936, cp950 …
  • mountpoint automatical assignment
  • large inode size: 128, 256, …
  • CIFS sharing over network
  • htree directory indexing

Article sponsored by Ian, owner of the site 1tb external hard drive

Popular Posts:

flattr this!

  28 Responses to “The Best Filesystem for an external hard disk of 1TB with cross platform support.”

  1. The NTFS filesystem reacts badly to sudden disconnections of external drives. USB drives so often get yanked out or powered down without going through a formal dismounting, leading to problems. If you can guarantee you won’t do this, then NTFS is fine; otherwise, FAT32 remains the only real cross-platform choice.

  2. Some other points to consider:

    Do you want to boot off the disk? Useful if you have a non-working or virus-infected computer to use. In which case it’s useful to create a small partition to boot from. 1GB would let you install a live-cd system, 5-8 GB lets you install a full Linux system. Windows can take more space, and anyway tends to be less device-independent.

    Do you want confidential files on the disk? You could create a Truecrypt virtual encrypted disk within a file on your big partition, but make sure the partition type allows files of that size. No good having that 10GB virtual disk on a FAT32 file system. Or you might allocate a partition for it.

    What kind of partition table do you want? Most systems understand Intel MBR format. But you then won’t be able to boot a Mac from it, they only boot from GPT partition tables – which are somewhat backwards compatible with MBR.

    What kind of disk geometry is implied in the partition table? If you have an “Advanced format” hard drive with larger blocksize – or a solid-state drive – then you really want any partition start blocks (including the first partition on a single partition drive) to have appropriate alignment to the physical sector size. Otherwise performance will be poor.

  3. Recently I formatted a 16Go USB key in exFAT. It is recognized by windows XP (work box), Mac OS10 (my wife’s computer) and my trusty Linux Mint laptop. I use it mostly to transfer iso files of DVD I have (we do not own a TV and watch films on our computers).
    Works well for me!

  4. I got external drive with network support couple of years ago. It has built-in controller with some kind of raid massive, I suppose. I really don’t care which filesystem it uses.
    I connected it to my wireless router and use it as external storage via Samba. It perfectly works from Windows, and it works from Linux if smbfs is installed.
    Actually, connection to that drive is routine part of reviews on my blog.

  5. limit size seems totally miscalculated … or you have very bad source of informations.
    for example NTFS file size limit is 16EB but partition limit is 2 TiB ? thats weird a bit :)

    File size limit:
    FAT32 : 4GiB
    NTFS : 16 EB
    ext3 : 2 TiB
    ext4 : 16 TiB

    Partition size limit:
    FAT32 : 2TiB
    NTFS : 2TiB
    ext3 : 32TiB
    ext4 : 1 EiB

    • You are right, the partition depend on the size of the blocks, i’ve took the starting max for ntfs, that is not the max overall:

      Maximum volume and partition sizes start at 2 terabytes (TB) and range upward. For example, a dynamic disk formatted with a standard allocation unit size (4 KB) can have partitions of 16 TB minus 4 KB. For more information about maximum volume and partition sizes, see the Windows Server 2003 Resource Kit, Server Management Guide at the Microsoft Windows Resource Kits Web site.

      http://technet.microsoft.com/en-us/library/cc779002(WS.10).aspx

      • Note that disks larger than 2TiB, and partitions larger than 2TiB, require GUID Partition Table (GPT) support, as MBR partitioning breaks at 2TiB.

  6. If considering NTFS, one should consider NTFS-3G Advanced from Tuxera.

    http://www.tuxera.com/community/ntfs-3g-advanced/

  7. Ext3 is a portable (win/linux) read/write solution. Ext4 is currently read-only on Windows. Ext3 easily handles large filesystems, much better than FAT. Create a small FAT partition which you can use to carry your portable tools. Make the rest ext3/4 (based on r/rw needs).

    http://www.ext2fsd.com/
    http://roshanbook.wordpress.com/2011/10/31/three-ways-to-access-linux-partitions-ext2ext3-from-windows-on-dual-boot-systems/

  8. For me if you’ve got a mixed environment with Linux, Mac and Windows, a NAS drive is a better investment than a USB drive, since you can have the NAS formmatted in ext3 or ext4 and still access with all OSs, since the clients do not interact directly with the filesystem.

  9. I use NTFS on both external drives and USB Flash Drives

    Works well and is compatible cross-platform with Linux, Windows, and Macintosh

  10. NTFS on Ubuntu (10.04 LTS, maybe others) via FUSE is a major PITA on a multiuser machine as it will only mount as writable by the desktop user at the time, others are locked out and no known fstab magic can fix it; a remount by root is required.

  11. ext4 works great on every operating system worth using.

  12. “A possible solution it’s to split your hard drive”

    Why does the author think the word “it’s” means “is”?

    He/she does this multiple times in the article and reading this is like finding a piece of grit in your food.

    • Thanks for the feedback John, probably the translator was drunk, i’ve corrected some (all ?) of these errors ;)

      Best Regards

      • When I bought my 1TB external drive, I also choosed NTFS (I didn’t read this article at this time). My main preoccupation was to have something very portable, and still reliable, with fast speed (eSATA).

        That over the alternative of a small fat 32 partition with win software to read ext along with a huge ext3 partition.

        So, after two years of use, I can say I’m very happy with that solution that has enabled me to take this tool around to flawlessly gather pictures, movies, documents, backups, … Between several operating systems, and also on a seedbox (Freebox).

        The ntfs drivers under linux are now working flawlessly in read/write, so until win is able to natively read ext fs, I guess ntfs is nice and works with hotplugin without problem.

        The only problem is in the case of accidental extinction without unmounting the FS. In this regard, my linux doesn’t do an automatic fs check once connected after hard extinction, and that may be an annoyance… But this is also the same for external ext formatted disks…

  13. Nice article – this topic is of concern to many non-technical people and technical people.

    You have provided the technical details herein but you could benefit from providing a real-world social example.

    Example – Movies – These files are ISO files and can reach typical sizes of 4.7 Gigabytes (movie copy programs scale down copied movies from dual-layer to [8.5 gigabytes] to single-layer [4.7 gigabytes]. Therefore – forget about Fat32 filesystems (4 Gigabyte file size limitation). A group of friends will share their movie collections an a USB-connected 1 TB drive that they pass around amongst each other. They run Windows, OSX, and Linux. Nobody wants to run special drivers for a filesystem. Solution = NTFS. And even if your movie copy program results in a 1-for-1 copy then your ISO files will be 8.5 gigabytes. Solution still = NTFS.

    Sharing songs and sharing books and sharing software will yield files that are smaller than ISO movie files so the above example is the only one that really matters.

    Case closed – NTFS wins.

    The various ext filesystems are fine for technical people but do NOT forget about the non-technical people who populate most of the computer-using public.

    • Thanks for these additional info Fyodor

    • As expressed in previous comment, those strongly advocating NTFS appear to be Windows centric and have little knowledge or understanding of other, more universal and robust File systems, as per comment by “Fyodor”.

      The ZFS file system can support all the file types as NTFS and more. One of several other advantages of ZFS over NTFS is CRC, and support for very large files – up to i Exabyte. Using external USB drives that one” passes around” is anachronistic – so twentieth century. A Network Attached Storage (NAS) based solution is far superior – far more flexible and can even be accessed over the Internet, or via WiFi access point.

      It really is important that Microsoft Windows followers understand that there are many great technologies available today, outside their very desktop myopic view, especially with Internet and mobile networking – in which Redmond has a very small and insignificant position or influence.

      • Re: NTFS-centric == Windows centric

        I would disagree with this position, simply from experience. I’ve been a Linux user and advocate since RHL 3.0.3 was the new kid on the block in 1996 (while I installed SLS 1.0 from floppy tape years before, I really got into things after RHL 3.0.3). I have found that Linux is far more forgiving of ‘alien’ filesystems than Windows is; if I’m going to interchange large files with a Windows user I’ll use NTFS, simple as that. And I don’t ever boot into Windows, either; I dual-boot OS X and Linux.

        Re: ZFS suitability for this specific task.

        While NAS (and SAN; have 400TB of fibre-channel SAN here) has its place, its place is not in simple file portability as related by the title of this blog entry. ZFS is overkill for a simple USB drive, which, although it might be ‘so twentieth century’ is still used, and is the question addressed by this blog entry.

        ZFS is fine technology; it’s just not appropriate for this particular use case.

        Re: Everything in the cloud.

        If I’m going to move or copy 1TB of data between two machines, and want it done sometime this week, I’m not going to use any cloud service to do it. USB 3.0, eSATAp, and even USB 2.0 and FW800, blow 100Mb/s ethernet away, and even gigabit ethernet has difficulty keeping up. Forget WiFi even keeping up; even USB 2.0 is going to blow its doors off.. Using NAS with NFS or Samba as the serving technology has overhead that further slows things down, even while they abstract away the filesystem used. And using NAS over typical US broadband? You might as well be doing things with RFC1149.

        The cloud has its place; this isn’t it. If I want to copy a few GB or even a few tens of GB, perhaps, but even then the typical broadband connection’s upload speed is slow as molasses at the South Pole, with 3G and 4G not much better, and costing per GB no less. Further, I may not have networking of any kind when I want or need to interchange. Cloud quits where network isn’t.

        And I may not want (or can’t put for legal reasons) this data in the cloud.

  14. Paragon Software sells several filesystem drivers for Linux, Mac OS X, and Windows. Seagate 1TB 2.5 inch externals have come with a ‘locked’ version of the Mac Paragon NTFS drivers; the full version isn’t too expensive and gives good results. Paragon has:
    Linux NTFS and HFS+ (journaled) drivers;
    Mac OS X NTFS and Ext2/3/4 drivers;
    Windows HFS+ (journaled) and Ext2/3/4 drivers;
    all with full support.

    I’m dualbooting Mac OS and Linux, and am using the Linux HFS+ and NTFS drivers and the Mac Ext2/3/4 and NTFS drivers for full drive compatibility.

    I had no end of problems with the in-kernel (not the Paragon) Linux HFS+ drivers, most notably the lack of journal support and very bad corruption when deleting large numbers of files. So far haven’t had any issues with the Paragon drivers.

    If read-only support of NTFS in Mac OS X is all that’s required, that’s built-in in 10.6.x, no extra drivers required; write capability needs drivers.

    For open source versions, look for ‘MacFUSE’ and its ext2/3 driver and NTFS driver.

  15. All those commenting who advocate NTFS are obviously Windows only users, and do not realize the world has moved on in regards multitude of Media server
    requirements. One worthwhile suggestion was to have server based on Network Attached Storage (NAS), and it would be more reliable, flexible and powerful if supported on ZFS File System that supports each of the Apple (iOS, HFS), Microsof (Fat32, NTFS), Linux (all) and Unix files, as well as ISO images in their native format.

    Not only does ZFS have CRC, Copy on Write, 128bit and SnapShots, it can support files up to 1 Exabyte and more.

    How cool is that!

  16. One would hope that the realization that this is an on-going problem would spur further work on developing open source OS-specific drivers for the various file systems. Personally, I would like to see better ext2/3/4 support for the OSX environment and better HFS+ support on the Linux side. Paragon does seem like a good solution, but really support should be transparent.

    • While support really should be transparent, in reality that is very difficult. Each OS vendor has its own reasons for the filesystem(s) it can use, and not use. None of the commercial OSes has a good reason to support the other’s filesystem, and very good fiscal reasons to not support open filesystems; this is the reality. Not what I would like to see, but reality.

      And ZFS has its own issues, legal and otherwise.

      I would presonally like to see SGI’s XFS used for interchange due to the very large filesystems that are supported, but that also has issues.

      The one filesystem that all of these OSes support in one form or another is UDF. While that is often thought of as the DVD filesystem, it should be able to be used on hard disks, although I’ve not personally tried it.

      • I’ve just been experimenting with it, and it’s very early days for me (seeing I’m not a OSX/Unix guy). I have a lapbook pro with OSX and dual boot windows 7. The UDF file system is installed in one of several partitions of my external hard drive and I can write to and read from both systems. I do believe there is a 2TB limit though- not a problem for me, but I believe no way near as much as ex-FAT or NTFS . The problem I have is that the UDF partition must be manually mounted from the terminal session in OSX, but mounts automatically as a drive in windows 7.

  17. The introduction mentioned “…or connect it to a TV” but in the article there was no indication give as to which formats are suitable for TV’s.

    Would be great if some of you could collate a bit of tech infos for the various manufactures. Their own documentation never mentions which filesystem formats are even supported. Often a ‘trial and error’ to find something that works.

    I would like portability between Win/Mac/Linux and my TV.

  18. NTFS looks like a good choice, but do NOT FORGET it is not case sensitive, if you use it for backing up an ext partition, etc. Happened before, and will happen again … unfortunately …

  19. I’ve got a solution that isn’t as impractical and exotic as ZFS on NAS or hindered with gotchas as NTFS (Linux can’t replay the journal if there’s an unclean unmount) or FAT32 (files size): UDF! Heck, it’s name is UNIVERSAL Disk Format for a reason! :-) This is commonly used on DVDs and other optical disks. Windows XP needs a third-party utility to write to it (it can read it), but Windows Vista on up, the last several OS X versions and Linux (particularly 2.6.30+) can all read and write to it just fine. It’s also resilient since it was designed for optical disks. It’s a vendor-neutral standard, so it’s properly documented, no vendor lock-in, etc. It has the same 16 EB file size limit of NTFS and a decent number of advanced features.

    UDF is a file system readable across a wider range of OSes than the other options (even read support of its older versions available in Win95 and Win98, BeOS, BSD, etc.) without the same drawbacks. I think it’s the logical choice.

 Leave a Reply

(required)

(required)


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>