ScoutFS is a clustered in-kernel Linux file system designed for very large archival storage systems.

It takes a different approach from putting an archive application on top of a conventional file system and repeatedly walking an enormous namespace. ScoutFS provides archive-oriented metadata and interfaces that let archive agents find and maintain files more directly. Multiple hosts can mount the same file system while sharing access to its underlying block devices, allowing a deployment to add nodes as archival bandwidth requirements increase.

Its design is aimed at installations where both file population and storage throughput can become extreme, while applications still see familiar file-system semantics. ScoutFS is therefore a specialist option for large data repositories and active archives rather than a general-purpose desktop file system.

This is free and open source software.

• Provides fully consistent POSIX semantics between participating cluster nodes.
• Uses atomic transactions to keep persistent file-system structures consistent.
• Stores archival metadata directly in the file system, avoiding the need to synchronise equivalent state with a separate external database.
• Separates resources dynamically so different nodes can perform writes in parallel with less contention.
• Uses a 64-bit design throughout, avoiding smaller integer limits on file sizes, directory sizes, and file counts.
• Supports multiple mounts on hosts that share access to the file system’s block devices.
• Keeps metadata and file data on separate block devices, allowing storage layouts to reflect their different I/O characteristics.
• Supports POSIX access control lists.
• Includes quorum membership, leader election, and fencing mechanisms to coordinate cluster operation and isolate failed mounts safely.
• Offers data preallocation controls for improving extent layout with large streaming writes.
• Includes user-space administration utilities for creating, inspecting, and operating ScoutFS installations.

Website: github.com/versity/scoutfs
Support:
Developer: Versity
License: GNU General Public License v2.0

ScoutFS is written in C. Learn C with our recommended free books and free tutorials.

Read our verdict in the software roundup.

Explore our carefully curated directory of recommended free and open source software, covering every major software category.

The directory forms part of our extensive collection of articles for Linux enthusiasts. It includes hundreds of detailed reviews, together with free and open source alternatives to proprietary software from companies such as Google, Microsoft, Apple, Adobe, IBM, Cisco, Oracle, and Autodesk.

LinuxLinks also covers interesting projects worth exploring, Linux-compatible hardware, free programming books and tutorials, and much more.

Know a useful free and open source Linux application that we haven’t covered? Tell us about it using our submission form.