Heroku like Elixir deployment on Digital ocean.

Recently building few microservices and apps in Elixir and deploying it to heroku, we realised that heroku was not very cost effective for us and also we can't get into every nuts and bolt of how heroku manages our app. As a part of learning things related to deploying to our own server and to save some bucks we decided to look for other options and we decided to experiment deploying one of our app to Digital ocean. One thing we were worried about, was whether we can make deployment as easy as heroku makes it for us. After some searching over internet, we landed at github repo named Gatling. Gatling is an awesome package which makes deployment to own managed server as simple as heroku deployment. Using Gatling we just have to push things to remote repo in our server and it takes care of rest of th rest. Will come back to awesomeness that Gatling provides out of the box shortly. Lets look how to initially setup DO and some basics on how elixir deployments actually works.

Initial Digital ocean setup

First step that we have to follow is register at DO and follow following guide -> Initial setup for Digital ocean

You can use steps mentioned in Initial setup guide to create a non root user named 'deploy', which we will use for deployment. This user will require a password when running the command with sudo and as gatling runs some sudo commands, we would have to allow this user to run sudo command without password. We can do so by editing the sudoers file using visudo command.

sudo visudo -f /etc/sudoers.d/deploy

Add following line to the file so that deploy user can run sudo command without password.

deploy  ALL=(ALL) NOPASSWD:ALL

You can also edit /etc/sudoers.d/deploy file using nano or any other editor, but it is recommended to be edited using visudo command. visudo command provide some additional feature which is very usefull to edit such sensitive files. visudo edits the sudoers file in a safe fashion. visudo locks the sudoers file against multiple simultaneous edits, provides basic sanity checks, and checks for parse errors.

How Elixir deployment works

Now with setup done, lets see how elixir deployment work. Elixir deployment is a two step process. First step is to build the release and then is to deploy it to the server. There is various packages available to build a release. One of the famous package for building the release is Distillery and recent versions of gatling uses it to build the release. What does release process does is it creates a single package which can be deployed anywhere, independently of an Erlang/Elixir installation. Once release is build we can copy it to server and we don't have to worry about installing anything, it would work out of the box. Every release is bundled with all the required package along with Erlang and Elixir. One think to note though is that your build environment should be same as environment wherein you will deploy the release so that the your release package works. You can't build the release on 64 bit system and expect it to run on 32 bit system.

Gatling does not handle building the release, it delegates it's task to Distillery. To make your elixir app capable of creating a release with Distillery, you need to add following line as dependency.

{:distillery, "~> 1.4", runtime: false}

For more information on Distillery, check this link

Other necessary Installations

You can install nginx using following command.

sudo apt-get install nginx

You would also have to install git on your server so that we can create a repo on server and push things there from our local.

 sudo apt-get install git

We would also have to install Erlang, Elixir on our server as we would be using same server for build and deploy. We would also have to install nodejs so that we can compile assets using brunch. Use following command to install Erlang, Elixir and nodejs.

wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb && sudo dpkg -i erlang-solutions_1.0_all.deb && rm erlang-solutions_1.0_all.deb
sudo apt-get update
sudo apt-get install -y esl-erlang elixir
curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -
sudo apt-get install -y nodejs build-essential

If you are using postgres as a database, follow this link to install it Install postgres in Do

prod.exs settings

config :my_app, MyApp.Endpoint,
  http: [port: {:system, "PORT"}],
  url: [scheme: "http", host: "myapp.com", port: 80],
  cache_static_manifest: "priv/static/manifest.json",
  # configuration for the Distillery release.
  root: ".",
  server: true,
  version: Mix.Project.config[:version]

config :my_app, MyApp.Endpoint,
  secret_key_base: System.get_env("SECRET_KEY_BASE")

config :my_app, MyApp.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: System.get_env("DB_USER_NAME"),
  password: System.get_env("DB_PASSWORD"),
  database: System.get_env("DB_NAME"),
  pool_size: 20

We can set above environment variable in /etc/environment file. Copy this lines in environment file.
MIX_ENV=prod
DB_USER_NAME=db_name
DB_PASSWORD=db_password
DB_NAME=db_name

Remember to open /etc/environment file with sudo command in editor of your choice as it won't be editable without sudo command.

Note: we don't need to set 'PORT' and environment variable it is set by Gatling when it runs.

Deployment using Gatling

mix archive.install https://github.com/hashrocket/gatling_archives/raw/master/gatling.ez

If you are setting up a brand new project, you would have to create a git repo on your server using this command.

mix gatling.load my_app

Once above command is ran, we get a git repo on server where we can push things and rest things would be handled by Gatling. Url of repo would be in following format remote_server_username@ip_address_of_server:<mix project name>, where remote_server_username would be 'deploy' in our case.
To push to this repo from local add it as a remote repo url using following command.

git remote add  production remote_server_username@ip_address_of_server:<mix project name>

For gatling to configure nginx to respond to particular domains, it needs file named domains in root of your project.
Check example of domains file from here. You can add all the domains to which you want nginx to forward your request to your app. You can add any domain if you don't own any during setup and change /etc/host file on you local to use this domain name.

Gatling support various callbacks that executes during different times while deployment is in progress. We need to use some of this callbacks to run important things like migrations automatically, so that we don't have to ssh to server just to run the migration. There are two files that we can add to root of our project deploy.exs and upgrades.exs.

Contents of deploys.exs

defmodule MyApp.DeployCallbacks do
  import Gatling.Bash

  def before_mix_digest(env) do
    # create static folder to have compiled asset
    bash("mkdir", ~w[-p priv/static], cd: env.build_dir)
    bash("npm", ~w[install], cd: env.build_dir)
    # compile static assets
    bash("npm", ~w[run deploy], cd: env.build_dir)
  end
end

Contents of deploys.exs

defmodule MyApp.UpgradeCallbacks do
  import Gatling.Bash

  def before_mix_digest(env) do
    bash("npm", ~w[install], cd: env.build_dir)
    bash("npm", ~w[run deploy], cd: env.build_dir)
  end

  def before_upgrade_service(env) do
    bash("mix", ~w[ecto.migrate], cd: env.build_dir)
  end
end

You can commit all this file and then push your branch to your newly created repo in server. And it's easy as we are pushing to any other repo.

git push production master

In above code production points to remote_server_username@ip_address_of_server:, which we have already add with 'git remote add' command.

For initial deploy you have to run one command from server to build release and deploy it. ssh to your server and run following command.

sudo --preserve-env mix gatling.deploy {project_name}

You don't have to run above command for subsequent deployment. Above task would do following things

• Install all mix dependencies
• Compile entire app
• Build release
• Create and migrate db
• Deploy app and start it.
• Configure nginx

Once above task is ran, our app should be up and when we go to domains mentions in domains file and if DNS is pointed properly to DO name server. If you don't own the domain that you have used in domains file, you can still access the site using the domain name in your local machine. Edit /etc/hosts file to add the domain name to point to the ip of Digital ocean server.

And thats it! We are done with deployment.