Recently, I was tasked with building a search feature for a database of videos. It was something I'd never previously done before and was pretty eager to learn in the process. But just like most problems in coding, I quickly realized there are a thousand different solutions for one challenge. The goal is to find the solution that closely fits your needs and requirements, while also being efficient. In this article, I'll attempt to build a search feature from scratch using several different approaches, along with a final suggestion should you want to delve any deeper.

Let's jump straight in and use a boilerplate website to search movies from an IMDB database. We'll first do a search using simply ActiveRecord and then we'll try using a gem called PG Search to do the same. Finally, we'll delve into Multisearch and Elastisearch, which are geared more for professional and large-scale websites.

We can start by generating the models:

rails g model director first_name last_name
rails g model movie title year:integer synopsis:text director:references
rails g model tv_show title year:integer synopsis:text

rails db:migrate

After we db:migrate and check our models, we just need to And add has_many :movies to Director since that wasn't auto-generated.

Rather than spending a ton of time creating new movies, here's code to generate a proper seed file. Going through line by line, we can see that it will take this yaml file with movie information, convert it into a ruby object, iterate over each field (director, movie, show) and create the object. Of course, make sure to rails db:seed afterwards

require "open-uri"
require "yaml"

file = "https://gist.githubusercontent.com/juliends/461638c32c56b8ae117a2f2b8839b0d3/raw/3df2086cf31d0d020eb8fcf0d239fc121fff1dc3/imdb.yml"
sample = YAML.load(URI.open(file).read)

puts 'Creating directors...'
directors = {}  # slug => Director
sample["directors"].each do |director|
  directors[director["slug"]] = Director.create! director.slice("first_name", "last_name")
end

puts 'Creating movies...'
sample["movies"].each do |movie|
  Movie.create! movie.slice("title", "year", "synopsis").merge(director: directors[movie["director_slug"]])
end

puts 'Creating tv shows...'
sample["series"].each do |tv_show|
  TvShow.create! tv_show
end
puts 'Finished!'

You can see that it worked by using rails c and checking movies.count.

Next we'll set up our routes.

Rails.application.routes.draw do
  root to: 'pages#home'
  resources :movies, only: :index
end

Which means we need a controller.

rails g controller Movie

Now, we define our index. We'll start with showing every movie in our database and then proceed to only search for the ones we want.

class MoviesController < ApplicationController
  def index
    @movies = Movie.all
  end
end

Next we can create a view file for our movie. Inside our movie folder we create a index.html.erb and we can have a basic html/bootstrap set up for a basic view page.

<div class="container">
  <div class="row justify-content-center">
    <div class="col-sm-8">
      <div id="movies">
        <% @movies.each do |movie| %>
          <h4><%= movie.title %></h4>
          <p><%= movie.synopsis %></p>
        <% end %>
      </div>
    </div>
  </div>
</div>

Lastly, for the setup we just need to set up a link for our homepage. If we run our server we can see a button that leads straight to our home page.

We can see a basic list of all our movies without having done any search yet.

PLAIN ACTIVE RECORD

We'll start by using plain ActiveRecord without using any other external or third party gems. We'll do it from scratch using what's given to us by Rails (with just a little bit of SQL).

To start, we need a form to type the query. We'll put it directly above our movie list.

<%= form_tag movies_path, method: :get do %>
  <%= text_field_tag :query,
    params[:query],
    class: "form-control",
    placeholder: "Find a movie"
  %>
  <%= submit_tag "Search", class: "btn btn-primary" %>
<% end %>

Here's what it now looks like:

A quick glance at the code shows that if we search something in the form, it should reload the same page. We'll need to code the functionality that returns the correct query.

If we search for "Superman", we know that it's a params in our code and also in our url. We can use this for our search. Let's start by changing our MoviesController. Instead of returning all movies, let's return only those with an exact match to the title.

class MoviesController < ApplicationController
  def index
    if params[:query].present?
      @movies = Movie.where(title: params[:query])
    else
      @movies = Movie.all
    end
  end
end

Here's what we get back if we search "Superman":

Pretty straightforward. But if we did lowercase "superman" we won't get anything in return. Additionally, if we have nothing in the search bar but press enter, we get nothing. That's not really useful behavior either. Let's change both of these.

class MoviesController < ApplicationController
  def index
    if params[:query].present?
      sql_query = "title ILIKE :query OR synopsis ILIKE :query"
      @movies = Movie.where(sql_query, query: "%#{params[:query]}%")
    else
      @movies = Movie.all
    end
  end
end

We can use an if statement to check if the query is present, then return the query. Else, return all movies. We now have a slightly better search, but should still have a more dynamic way to search for Superman. Luckily, there's a way in ActiveRecord to do case insensitive searches.

This line here:

@movies = Movie.where(title: params[:query])

is completely identical to this line:

@movies = Movie.where("title LIKE ?", params[:query])

The only difference is that it uses SQL syntax. But in SQL, ILIKE is case insensitive. So we can replace the above line with:

@movies = Movie.where("title ILIKE ?", params[:query])

But what if we don't want to search only by title? Let's say I know of a movie that takes place in Gotham City but I just can't remember the name. Such a word might be found in the synopsis of the film but not the title.

@movies = Movie.where("title ILIKE ? :query OR syllabus ILIKE :query", query: "%#{params[:query]}%")

As you can see, we add the OR keyword to look through the synopsis. But with the above syntax it also checks to make sure that the synopsis CONTAINS the keyword. Or more clearly, does the synopsis have the word Gotham anywhere inside of it? Now if we search for "Superman", we still get the title. But we also get all movies with "Gotham" anywhere the synopsis.

That's great progress! But it's still not perfect. Although we have "Batman V Superman: Dawn of Justice" in our database, when we type in batman superman, we get nothing returned.

But why? The problem is that it's looking for instances when "superman" comes directly after "batman". The problem is the "v" in the title. To fix this, we can bring something in from PostgresSQL: full-text search. The keyword we can use is @@.

@movies = Movie.where("title @@ :query OR syllabus @@ :query", query: "%#{params[:query]}%")

@@ is the query for full text search. Say for example we have the word jump. Full text search won't simply search for words containing the letters "jump" like the LIKE keyword. It will go further and grab any words syntactically associated with that word in the English language. Jumped, jumping, jumper, etc will be returned. You can also pass it multiple words and it will search for each of them individually.

Now when we update it with a full-text search, it will search for each of them to be somewhere in the text and not necessarily directly next to each other as they are in the input field. To drive the point home about the power of full-text search, we can even type the word "fears" and it will match the first word of the synopsis "fearing" too.

We're super close to getting a great search by only using ActiveRecord! BUT, so far, we are only searching the movies table. What if I know the director, but have no idea the name of their films?

So far, our program doesn't allow for searches on another table. We also want to search through the associations in the database. The magic word for this that no one really like is JOIN. We'll have to join the table in order to search through both associations.

Here's the query that we're going to use:

class MoviesController < ApplicationController
  def index
    if params[:query].present?
      sql_query = " \
        movies.title ILIKE :query \
        OR movies.synopsis ILIKE :query \
        OR directors.first_name ILIKE :query \
        OR directors.last_name ILIKE :query \
      "
      @movies = Movie.joins(:director).where(sql_query, query: "%#{params[:query]}%")
    else
      @movies = Movie.all
    end
  end
end

We want to search for a movie's titles OR the movie's synopsis, OR the director's first OR last name. In order to do all this, we simply need to join the table at the beginning of the query.

Let's try again:

Although it's still not 100% perfect (we can't do a successful search for "Batman Nolan"), it's pretty good considering we didn't use any external libraries!

PG Search

PG Search is a wonderful gem that does a lot of the heavy lifting on your computer for you.

The documentation is here. It's well-written and straightforward. I'll add the gem to my gem file, stop my server, and bundle install.

Now we will add some code to tell PG Search how to work in our model instead of our controller. And in our controller, we will just call the method from the model.

class Movie < ApplicationRecord
  # [...]
  include PgSearch::Model
  pg_search_scope :search_by_title_and_synopsis,
    against: [ :title, :synopsis ],
    using: {
      tsearch: { prefix: true } # <-- now `superman batm` will return something!
    }
end

The line that reads include PGSearch just brings the gem into our model. The next line, pg_search_scope is the method that you will later call when you do the search. We have the freedom to name it whatever we want. For our test, we named it search_by_title_and_syllabus. In against we'll tell which columns we will search in. And the last line we see tsearch (the same as "@@") and prefix which allows for partial words (e.g. "batma" would still match for "batman").

So instead of our previous code with ActiveRecord, we can now use our search_by_title_and_syllabus method.

# rails c
> Movie.search_by_title_and_synopsis("superman batm")

Let's see if it still works

The setup for PG Search is incredibly easy. We already have what took 10 lines of code in one method.

You can still search with associated models just by appending the code to add associated_against, like so:

class Movie < ApplicationRecord
  belongs_to :director
  include PgSearch::Model
  pg_search_scope :global_search,
    against: [ :title, :synopsis ],
    associated_against: {
      director: [ :first_name, :last_name ]
    },
    using: {
      tsearch: { prefix: true }
    }
end

Cool. But one problem you may have noticed. We still have a TV Show table that's not even included in any search. What do we do?

Multi-Search

As the name suggests, multi-search allows us to search against more than one model. It's a fairly advanced way to go about doing a search so I will keep this section brief. But even if you want to do a search on multiple models, it still needs one table to search for. What it will do is grab all the information it needs and compile it into one table and then use that against all its searches. For that we'll need to do some setup, but it's relatively simple.

rails g pg_search:migration:multisearch
rails db:migrate

class Movie < ApplicationRecord
  include PgSearch::Model
  multisearchable against: [:title, :synopsis]
end

class TvShow < ApplicationRecord
  include PgSearch::Model
  multisearchable against: [:title, :synopsis]
end

# rails c
PgSearch::Multisearch.rebuild(Movie)
PgSearch::Multisearch.rebuild(TvShow)
results = PgSearch.multisearch('superman')

results.each do |result|
  puts result.searchable
end

This will create a brand new table in the database called pg_documents and when you run the multisearch command, it will run a search on everything in that newly created table. There's a ton more to it. I would suggest reading the documentation and adjust what you learn about it to your needs. But no matter what you are trying to do with a search engine and no matter how advanced, chances are that Multi-Search can help.

Elasticsearch

Why would you need to use Elastisearch? There are a couple of key features about Elasticsearch that make it an ideal gem to use:

• It gives suggestions
• It understands misspellings
• It allows you to boost some fields in search through scoring
• It has autocomplete
• It's fully geo-aware

The infrastructure for this is slightly more complicated, but it's the same exact logic as we were doing for PG Search: we create a table, we add stuff to it, and we search on that table instead of our own. The difference with Elasticsearch is that the table is no longer in your database. It's on their servers, an external service.

# OSX/terminal
brew install elasticsearch
brew services start elasticsearch

# Gemfile
gem 'elasticsearch', "< 7.14" # to use a supported version of the Elasticsearch Ruby client
gem 'searchkick', '~> 4.0'

#terminal
bundle install

# app/models/movie.rb
class Movie < ApplicationRecord
  searchkick
  # [...]
end

# Launch a `rails c`
Movie.reindex
results = Movie.search("superman")

results.size
results.any?
results.each do |result|
  # [...]
end

The functionality is relatively the same as what we are doing before with the exception of the external servers. The main difference is that instead of creating a table in our database, when you re-index you send all of your movie information to the Elasticsearch server and when you search it returns the information you need. This turns out to be much faster than working through your own database, especially as your dataset gets larger and larger.

And there you have it. Three different ways to approach building a search engine with Ruby on Rails. Whether its for a small personal project or your working for a larger tech company in need of an efficient search option, I hope you found this article useful! :)