Upload files on S3 with Elixir/Phoenix using ex_aws

In one of our Rails 4 app, we decided to move file and image uploads to another microservice so that the load on server is reduced when a big file is uploaded. We decided to do this in Phoenix. In Phoenix, we have ex_aws package which makes file uploads to S3 very simple just like Rails. So lets get started. Add ex_aws Update mix.exs to include following dependencies. defp deps do [ ..., {:ex_aws, "~> 1.0"}, {:poison, "~> 2.0"}, {:hackney, "~> 1.6"}, {:sweet_xml, "~> 0.6"

Access files from sftp server

In one of the feature, we had to import xlsx which was exported from sqlite database dump. The rows had few image names which was uploaded to sftp server. We had to upload those to images on S3. Importing xlsx row data was simple but the challenge was how to upload images from sftp server to S3. This is where net-sftp gem came to our rescue (though its not managed anymore). To get this working, we need 4 things: host, username and password using which we are going to login to sftp server and path where images are uploaded. We

Method / Process to export active record data to excel file, save xls file on S3 and provide download xls feature

In one of our web application, we had a specific requirement wherein the client wanted the data from the app into a specific xls format. Thus we needed to export data from our app in this custom format and save the xls file on S3. Our app will then show a link to "Download" in ActiveAdmin UI clicking which it will start downloading this xls file from S3 on clients local PC. We have broken down the process into numerous steps, hope this will be helpful. The process goes as follows: The data from the app is exported

Backing up Databases and Directories

Backup your Database OR any of your personal stuff like directories with [backup][1] Backup's Components Archives and Databases Compressors and Encryptors Storages Syncers Notifiers Its a handy backup tool with plenty of storage options (for example: S3, Dropbox, FTP, SCP and many more) For installation(Dependencies should be installed if you are working with shell environment) gem install backup backup dependencies --install net-ssh backup dependencies --install net-scp backup dependencies --install excon backup dependencies --install fog Now execute the @backup generate@ command backup generate:model -t backup [-c ] --archives \ --storages='local,s3' --compressors='gzip' --notifiers='mail,twitter' This will create

Enabling encryption on S3

Files uploaded on S3 are not encrypted. To make them secure, we either need to encrypt the file at client's end then upload or configure S3 settings to encrypt the files after upload. i.e Server Side Encryption. Amazon S3 Server Side Encryption (SSE) employs AES-256, an encryption standard that provides a considerably high level of protection. To allow SSE for S3, just include following code in headers while uploading file: 'x-amz-server-side-encryption' => 'AES256' To check, login to S3 and check properties of the file uploaded. The "Server Side Encryption" option under Details tab should show AES-256 radio