Streaming Decryption Over HTTP via aespipe

August 23, 2013

This is a small example of how you can store AES encrypted files on S3, Rackfiles or whatever, and stream them unencrypted.

The trick here is to stream to whole process, so the client requests a file which this proxy then streams to the client unencrypted.

You’ll need aespipe. Also this won’t work with WEBrick. You’ll need thin, puma, or some other server that supports streaming.

To encrypt a file:

aespipe <inputfile >outputfile.enc

Then upload it using whatever mechanism. I’m assuming that the encrypted file on the object store or wherever is postfixed .enc.

require 'sinatra'
require 'uri'
require 'net/https'
require 'thin'
require 'mime/types'

get '/:uri' do
  start_time=Time.now
  total_size=0
  uri = URI(params[:uri])

  filename = File.basename(uri.path).gsub(/.enc$/,'')

  # You can't set the Content-Type header after starting the stream.
  # So we try to guess it from the file extension. Another option is to
  # first make a HEAD request to the URI to get the mime type. This would 
  # also allow you to get the Content-Length and set it in the response
  # header so the user knows how big a file he is downloading.

  content_type MIME::Types.type_for(filename)[0] || "application/octet-stream"
  
  # This is only needed if you want to force the browser to "Save As" the file
  # Remove it if you wish to stream images that display in the browser.
  headers 'Content-Disposition' => "attachment; filename=\"#{filename}\""

  stream do |out|

    IO.popen("aespipe -d -P /somewhere/password_in_clear_text", 'r+') do |pipe|
      Net::HTTP.get_response(uri) do |res|
        total_size = res.header['Content-Length'].to_i
        res.read_body do |chunk|
          pipe.write chunk
          out << pipe.read_nonblock(100000) rescue nil # read at most 100KB at a time
        end
        # aespipe encrypts data in 512 byte block. It null pads data to reach a
        # multiple of 512. We must tell it there is no more data to read so it
        # can perform this operation and read the remaining data.
        pipe.close_write
        out << pipe.read_nonblock(100000) rescue nil # read at most 100KB at a time
      end
    end

    puts "#{uri.path} #{((total_size/1024)/(Time.now-start_time)).round(2)} KB/s"
  end

end
Share this post on Twitter
Morten Møller Riis

By Morten Møller Riis

I am a programmer, sysadmin, devops. I work for Gigahost in Copenhagen, Denmark. I am based in Odense, Denmark.

Twitter   ·   LinkedIn   ·   E-mail