S3FS : Mounting Amazon S3 as a filesystem

S3FS (Simple Storage Services File System) is a FUSE based file system backed by Amazon S3 storage buckets, and if mounted once, S3 can be used just like a drive in our local system. This helps a lot if you have an app which interacts very frequently with Amazon S3. Follow these steps to mount a bucket from s3 to local system using S3fs in UBUNTU 1. First of all you need to install the libraries that are needed by s3fs to run on the system. To install those libraries run the following command in console sudo aptitude install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev comerr-dev libfuse2 libidn11-dev libkadm55 libkrb5-dev libldap2-dev libselinux1-dev libsepol1-dev pkg-config fuse-utils sshfs 2. After that Download the appropriate version of s3fs you want to use by the following link In this example we are using s3fs-1.68, and can be downloaded from this link 3. After downloading the s3fs go to that path in your terminal and run the following commands
  • tar xvzf s3fs-1.68.tar.gz
  • cd s3fs-1.68/
  • ./configure –prefix=/usr
  • make
  • make install (as root)
4. After this you need to supply your s3 credentials to s3fs to mount the bucket. There are different ways of doing that, you can choose whichever is suitable to you
  • By setting the AWSACCESSKEYID and AWSSECRETACCESSKEY environment variables by using export command in UBUNTU. export AWSACCESSKEYID=******************* export AWSSECRETACCESSKEY=xxxxxxxxxxxx
  • By using a .passwd-s3fs file in your home directory. In this you just need to create the .passwd-s3fs file in home directory, and add the creds for s3 into it. The s3fs password file has this format (use this format if you have only one set of credentials): accessKeyId:secretAccessKey If have more than one set of credentials, then you can have default credentials as specified above, but this syntax will be recognized as well: bucketName:accessKeyId:secretAccessKey
  • By using the system-wide /etc/passwd-s3fs file. In this you just need to create the passwd-s3fs file in /etc directory, and add the creds for s3 into it similar to the last point.
5. After that change the permissions of the password file
  • If you are using ~/.passwd-s3fs then permission should be 600
  • If you are using /etc/passwd-s3fs then permission should be 640
6. And last, to mount bucket(mybucket) to specific directory(/path/to/directory/) s3fs mybucket /path/to/directory/ 7. To unmount the mounted drive use this command. fusermount -u /path/to/directory/ 8. To automount s3 bucket to every time to local system after boot, edit the /etc/fstab file and add this at the end of the file s3fs#bucket_name /path/to/directory/ passwd_file=/path/to/passwd/file And after this edit /etc/rc.local and add the following line before exit 0. mount -a 9. If you get any error/warning during system boot mounting just add the option in the current command s3fs#bucket_name /path/to/directory/ fuse _netdev,passwd_file=/path/to/passwd/file This command will omit the warnings and mount the bucket to your local system. Now you can access Amazon S3 just like another folder on your file system.
  • harini

    Great Article..It was very informative..I need more details from your side..include some tips..