(guest@joequery.me)~ $ |

Finding my cat Odin using Python and Elasticsearch

(This post is part of the finding odin series.)

Each time I have started a project, the motivation for doing so has either stemmed from

  • following my manager's orders so I can get a paycheck
  • wanting to mess around with some tech just for fun

But this project is different.

A couple of months ago, my 9-year-old cat Odin did not come home. He has been an indoor/outdoor cat his entire life. I unfortunately did not have him chipped. I thought I had the procedure ordered at his last vet visit but apparently it was not done.

The only guaranteed positive that comes out of every misfortune you encounter is the ability to truly sympathize with those facing the same misfortune. The words "my pet is missing" now invoke reminders of the guilt, the uncertainty, the longing, and the lack of closure that come from losing your companion. I now know the emotional torment hope can put a person through.

I've read very helpful guides on how to use social media, flyers, and various resources to increase the chances of finding a lost pet. The city of Austin has an invaluable Lost and Found database accessible via the Austin Animal Center (AAC) website. I check this page every day as part of my search process, as do the other people in Austin searching for their lost pet.

I have decided to use my development abilities to create a tool that can ever-so-slightly ease the burden of people looking for their pets in my city. I hope to accomplish this by improving the search functionality of AAC's Lost and Found Database.

Examining the AAC Lost and Found Database

The AAC's Lost and Found web page has an extremely basic filter.

Additionally, the "Main Color Grouping" option shows no results for the "Other" option, which is unfortunate since my orange tabby does not fit the Black, White, or Brown category.

Here is an example results page:

And an individual listing:

Project goals

My aim is to increase the utility of the data provided by AAC by implementing a more mature filtering system. I would also like to implement an alerting system that sends an email/sms when a pet has been found on the AAC website that matches user-defined criteria. I would also like to allow users to contribute in the search for a lost pet by creating and voting on appropriate search tags that would aid in the search.

Technology choices

The main components of this project will be Python for the language and Elasticsearch for the data store. Python has great HTTP client libraries and html parsing libraries. Elasticsearch describes a very similar use-case to our scenario right on their Getting Started guide, so it seems like a good fit for this project.

You run a price alerting platform which allows price-savvy customers to specify a rule like "I am interested in buying a specific electronic gadget and I want to be notified if the price of gadget falls below $X from any vendor within the next month". In this case you can scrape vendor prices, push them into Elasticsearch and use its reverse-search (Percolator) capability to match price movements against customer queries and eventually push the alerts out to the customer once matches are found.

The task list

At this point, our to-do list for this project consists of the following:

  1. Extract all the pet listings from the AAC Lost and Found Database using Python.
  2. Extract certain keywords from the listings and use them as tags for filtering
  3. Shove all of the pet listings into Elasticsearch
  4. Check for new listings on a periodic basis
  5. Allow for custom tags to be added to listings
  6. Create a basic front end

Features previously discussed will be implemented at a later time. The tasks listed above are enough to create a useful lost pet search engine for residents in the city of Austin.

The process

I plan on documenting my process for this project. I believe beginner and intermediate developers will find it useful to observe how a project evolves from conception, to prototype, to final product. The Finding Odin series listing will show all the blog posts for this project.

Tagged as python

(This post is part of the finding odin series.)

Date published - November 19, 2016