Heroku offers many features that make writing & deploying web applications extremely painless. However, their networking options make it moderately difficult to connect securely to resources in EC2. Thankfully, with a little bit of elbow grease you can use stunnel to create a secure network tunnel directly to a machine inside your VPC. This blog post will walk you through all of the necessary steps. 

 

Prerequisites 

  • An application running on Heroku 
  • An EC2 instance in a public subnet on EC2. You'll need the IP address for configuring stunnel. Additionally, your ACL or Security Group must allow inbound traffic from the internet on the port you choose for stunnel. This guide assumes you're running RHEL6 or CentOS6, but any OS that can run stunnel should be fine. File locations may vary.
  • A cert & private key in PEM format for the client & server 
  • This example is for for securing an Elasticsearch endpoint running on port 9200, but you may use it for any port of your choosing – you'll just need to change the parts in both stunnel.conf under [elasticsearch_api] to correspond to the service you're using. 

Your remote server 

First, let's configure your remote endpoint. This will give you a chance to test your stunnel setup before setting up the certificates & configuration on Heroku. 

Package installation 

Install stunnel and the ca-certificates package:

yum install -y stunnel ca-certificates

Configuration

Configure stunnel for your machine. You'll need your cert & ca installed in/etc/stunnel/cert.pem and /etc/stunnel/ca.pem. The block below configures stunnel to accept connections on :9201 and route them to 127.0.0.1:9200. Don't forget that whatever "accept" port you configure below must be allowed to connect in your security group or ACL setup in your VPC! 

setuid = stunnel4
setgid = stunnel4

debug = 7
output = /var/log/stunnel4/stunnel.log
pid = /var/run/stunnel4/stunnel.pid

cert = /etc/stunnel/cert.pem

verify = 2
CAfile = /etc/stunnel/ca.pem
CRLpath = /etc/ssl/certs

options = NO_SSLv2

[elasticsearch_api]
client = no
accept = 0.0.0.0:9201
connect = 127.0.0.1:9200

If you'd like, you can configure monit to monitor your stunnel as well, so that is resilient to failure. Example: 

check process stunnel with pidfile /var/run/stunnel4/stunnel.pid
  start program = "/etc/init.d/stunnel start"
  stop program  = "/etc/init.d/stunnel stop"

Your Heroku App 

This part assumes you're using Rails, but stunnel is not Rails specific. You'll need to replace this section with code applicable to the framework you're using. 

Install the stunnel buildpack on your Heroku app 

Follow the guide here to add the stunnel buildpack to your application. 

Add your cert, ca & key to your Heroku app's environment 

I recommend doing this from the command line to save yourself some pain. Note that CA variable should be your CA trust chain cat'd together, and your cert variable should be the private key used to sign the certificate and the certificate itself. 

cert=$(cat /path/to/my/cert)
cacert=$(cat /path/to/my/ca)
heroku config:set STUNNEL_CERT=”$cert”
heroku config:set STUNNEL_CA=”$cacert”
heroku config:set STUNNEL_ENDPOINT=your.remote.servers.public.ip.address
heroku config:set USE_STUNNEL=true

Write these certs to disk at boot time 

In rails, add this to config/application.rb. If you aren't using Rails, you'll need to do this with a mechanism that runs every time the dyno boots. I recommend adding a USE_STUNNEL env flag. 


unless File.exist?('/app/conf/ca.pem')
 puts "Writing ca for stunnel to disk"
 File.open('/app/conf/ca.pem', 'w') { |file| file.write(ENV['STUNNEL_CA']) }
end
unless File.exist?('/app/conf/cert.pem')
 puts "Writing cert for stunnel to disk"
 File.open('/app/conf/cert.pem', 'w') { |file| file.write(ENV['STUNNEL_CERT']) }
end

Configure Stunnel 

Drop this into conf/stunnel.conf. Remember that this example is for an elasticsearch endpoint running on port 9200, but you can change that block for whatever service you're running. Note the use of STUNNEL_ENDPOINT, which we will see later. 

service = stunnel-client
cert = /app/conf/cert.pem
CAfile = /app/conf/ca.pem
verify = 2
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
debug = 3
foreground = yes
session = 86400
TIMEOUTidle = 86400
client = yes
output = /tmp/stunnel.log

[elasticsearch_api]
accept = localhost:9201
connect = STUNNEL_ENDPOINT:9201

Test your stunnel 

Before moving on, test your stunnel. Copy conf/stunnel.conf to conf/stunnel2. Replace STUNNEL_ENDPOINT with your actual server, and point cert and CAFile to the actual location on your filesystem. Then, test your stunnel like so: 

stunnel conf/stunnel2.conf

If it works, you should be able to 'telnet localhost 9201'  (or whatever port you configured). If not, resolve your CA / Cert ? Config settings before moving on. 

Fork stunnel for each dyno that spins up 

Again, config/application.rb. Note that we replace STUNNEL_ENDPOINT in the config file with our environment configured one. 


f = File.read('/app/conf/stunnel.conf')
t = f.gsub('STUNNEL_ENDPOINT', ENV['STUNNEL_ENDPOINT'])
File.open('/app/conf/stunnel.conf', 'w')  {|file| file.puts t }
fork_stunnel = fork do
 puts "Forking stunnel..."
 exec "/app/vendor/stunnel/bin/stunnel /app/conf/stunnel.conf"
end
Process.detach(fork_stunnel)

Try it out! An easy way to test your stunnel is to remotely boot rails console (heroku run nails console) and attempt to connect to your remote resource. 

Special thanks to Steve Salevan for helping with this technique!

Header image "Tunnel" is copyright (c) 2009 by Michael Caven, published under a Creative Commons Attribution Share-Alike 2.0 License.

Comment