- Notifications
You must be signed in to change notification settings - Fork226
A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.
License
awslabs/mountpoint-s3
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Mountpoint for Amazon S3 is a simple, high-throughput file client formounting an Amazon S3 bucket as a local file system. With Mountpoint for Amazon S3, your applications can access objects stored in Amazon S3 through file operations likeopen andread. Mountpoint for Amazon S3 automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface.
Mountpoint for Amazon S3 is optimized for applications that need high read throughput to large objects, potentially from many clients at once, and to write new objects sequentially from a single client at a time. This means it's a great fit for applications that use a file interface to:
- read large objects from S3, potentially from many instances concurrently, without downloading them to local storage first
- access only some S3 objects out of a larger data set, but can't predict which objects in advance
- upload their output to S3 directly, or upload files from local storage with tools like
cp
but probably not the right fit for applications that:
- use file operations that S3 doesn't natively support, like directory renaming or symlinks
- make edits to existing files (don't work on your Git repository or run
vimin Mountpoint 😄)
Mountpoint for Amazon S3 does not implement all the features of a POSIX file system, and there are some differences that may affect compatibility with your application. SeeMountpoint file system behavior for a detailed description of Mountpoint for Amazon S3's behavior and POSIX support and how they could affect your application.To troubleshoot file operations that may not be supported by Mountpoint, see thetroubleshooting documentation.
Mountpoint for Amazon S3 is generally available! We're tracking future feature development on theMountpoint for Amazon S3 public roadmap. We're always interested in feedback on features, performance, and compatibility. Please send feedback byopening a new GitHub issue or adding your input to an existing roadmap issue.
Important
Mountpoint for Amazon S3 v1.4.0, released on January 26, 2024, contains anissue that causes intermittent read failures. We recommend that customers using v1.4.0 upgrade to v1.4.1 or later.
On Amazon Linux 2023, you can install Mountpoint for Amazon S3 directly from the repository:
sudo dnf install mount-s3On other RPM-based distributions (Amazon Linux 2, Fedora, CentOS, RHEL; excluding SUSE), run these two commands to install Mountpoint for Amazon S3 on your EC2 instance (for Graviton instances, replacex86_64 witharm64 in the URL):
wget https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.rpmsudo yum install -y ./mount-s3.rpmOn Ubuntu, use these commands instead (for Graviton instances, replacex86_64 witharm64 in the URL):
wget https://s3.amazonaws.com/mountpoint-s3-release/latest/x86_64/mount-s3.debsudo apt-get install -y ./mount-s3.debNote
SeeInstalling Mountpoint for Amazon S3 for detailed instructions and other installation options, including theMountpoint for Amazon S3 CSI driver for Kubernetes applications, or building from source.
Once you've got Mountpoint for Amazon S3 installed, you can mount your Amazon S3 bucket.You'll need valid AWS credentials to access your bucket.Mountpoint will automatically use credentials froman IAM role associated with your EC2 instance,or theAWS_ACCESS_KEY_ID andAWS_SECRET_ACCESS_KEY environment variables,or fromseveral other sources.
To mount your bucket, run this command,replacingamzn-s3-demo-bucket with the name of your bucketand/path/to/mount with the directory you want to mount the bucket to:
mount-s3 amzn-s3-demo-bucket /path/to/mountNow you can work with your bucket contents as if they were a local file system:
ls /path/to/mountecho "Hello World!" > /path/to/mount/Data.txtcat /path/to/mount/Data.txtWhen you're finished accessing your bucket, you can unmount it (you might needsudo):
umount /path/to/mountNote
SeeConfiguring Mountpoint for Amazon S3 for more details on how to configure and use Mountpoint.
Mountpoint for Amazon S3 is designed for high-performance access to the Amazon S3 service. While it may be functional against other storage services that use S3-like APIs, we aren't able to provide support for those use cases, and they may inadvertently break when we make changes to better support Amazon S3. We welcome contributions of minor compatibility fixes or performance improvements for these services if the changes can be tested against Amazon S3.
We welcome contributions to Mountpoint for Amazon S3! Please seeCONTRIBUTING.md for more information on how to report bugs or submit pull requests. We especially welcome contributions to issues tagged asgood first issues to work on.
For potential bug reports, seeLOGGING.md for details on how to capture logging data.For performance issues, seeBENCHMARKING.md for documentation about performance regression testing.
If you discover a potential security issue in this project we ask that you notify AWS Security via ourvulnerability reporting page. Please donot create a public GitHub issue.
This project has adopted theAmazon Open Source Code of Conduct. SeeCODE_OF_CONDUCT.md for more details.
This project is licensed under the Apache-2.0 License. It builds on a number of other awesome projects with open source licenses, primarily:
- Thefuser bindings for FUSE (MIT license)
- TheAWS Common Runtime for interacting with S3 (Apache 2.0 license)
Seedeny.toml for a list of licenses used by our dependencies.
About
A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.
Topics
Resources
License
Code of conduct
Contributing
Security policy
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Uh oh!
There was an error while loading.Please reload this page.