When dealing with image uploads in a web application, handling large files efficiently is crucial. Storing images in memory can lead to significant memory usage, especially with large or numerous files. These large files are waste of server resources, cause unnecessary computation, and if you read them into memory, can have great scalability concerns. Instead, you can stream images directly to AWS S3 using multipart/form-data to avoid memory bloat. This article will walk you through how to achieve this using Ruby on Rails.

Step-by-Step Guide

1. Setup AWS S3

First, you’ll need a way for your application to connect with S3, the destination of your images. First, ensure you have the aws-sdk-s3 gem installed and configured in your Rails application.

Add the gem to your Gemfile:

gem 'aws-sdk-s3'

Run bundle install to install the gem.

Configure your AWS credentials in config/initializers/aws.rb:

Aws.config.update({
  region: 'your-region',
  credentials: Aws::Credentials.new('your-access-key-id', 'your-secret-access-key')
})

2. Create an S3 Client

Create an S3 client instance to interact with your S3 bucket:

s3 = Aws::S3::Resource.new
bucket = s3.bucket('your-bucket-name')

3. Handling Multipart/Form-Data

Install the multipart-post gem to handle multipart/form-data:

Add the gem to your Gemfile:

gem 'multipart-post'

Run bundle install to install the gem.

4. Streaming Image to S3

Create a controller action to handle the image upload. Use the multipart-post gem to stream the image directly to S3 without loading it into memory.

require 'aws-sdk-s3'
require 'net/http/post/multipart'

class ImagesController < ApplicationController
  def create
    file = params[:file]
    upload_to_s3(file)
    render json: { message: 'Image uploaded successfully' }, status: :ok
  rescue => e
    render json: { error: e.message }, status: :unprocessable_entity
  end

  private

  def upload_to_s3(file)
    s3 = Aws::S3::Resource.new
    bucket = s3.bucket('your-bucket-name')

    # Create an object for the file
    obj = bucket.object(file.original_filename)

    # Stream the file to S3
    obj.upload_stream do |write_stream|
      file.tempfile.open do |read_stream|
        IO.copy_stream(read_stream, write_stream)
      end
    end
  end
end

5. Routing

Add a route for the image upload in config/routes.rb:

Rails.application.routes.draw do
  resources :images, only: [:create]
end

6. Testing the Upload

You can test the upload using a tool like curl:

curl -F "file=@/path/to/your/image.jpg" http://localhost:3000/images

Conclusion

By streaming images directly to AWS S3 using multipart/form-data in Ruby on Rails, you can handle large file uploads efficiently without consuming excessive memory. This approach ensures that your application is scalable, even when dealing with a large number of image uploads.