A Simple URL Shortening Service

Problem Statement

The customer requires a simple API-based service for a) shortening URLs (or otherwise encoding them) and b) retrieving the encoded/shortened URLs for lookup via a client browser (or bulk lookup tool ...).

(Don't ask me why such a thing would be useful - bit.ly springs to mind)

Let's begin with the high level architectural design and gradually drill down to implementation in code.

System Description

The System implements the following functionality:

• Accepts an HTTP/S POST  to shorten (encode) a URL from a browser client to an API endpoint.
• The API accesses functions to encodes the URL, stores it in a Redis database key-value table for later lookup
• A Separate API service accepts an HTTP/S GET request for the shortened URL, look-up the shortened URL in the Redis table and returns the decoded URL in an HTTP/S 302 REDIRECT response to the client browser. This allows the user client to access the decoded URL via http/s

The System design achieves the following non-functional requirements in the described way:

Architectural Diagram

Continuous Delivery Automation

To achieve Continuous Delivery or Continuous Deployment the following deployment pipeline is included in the design:
A Jenkins pipeline is defined and implemented using Jenkins declarative pipeline script to deploy the Infrastructure of the system as Code (IAC):

a) Terraform is used to create the AKS cluster, Redis cache service, Azure Load Balancer, DNS CNAME records. This code is versioned in a git repository
b) The Kubernetes Pod and ingress configuration is defined as Helm charts also stored in a separate Git Repo
c) Credentials and Secrets required by the service for accessing the Redis DB and other secure services are stored in a credential vault accessed by Jenkins pipeline to embed in the deployment. This ensures no secrets are stored in the git repos.
d) The Jenkins deployment pipeline itself is defined using either Jenkins Pipelinescript  or Groovy directly and kept as versioned code in Git.
e) "Continuous Integration" is implemented to improve quality of the deployment code by triggering a build of a development/staging system every time changes are made to a git master branch.
f) The Redis DB Schema is versioned git as code and deployed as a separate stage of the pipeline

Redis Database Table Schema

Shortened URL code field name
Actual URL Code field name
shortURL
trueURL

URL Shortening Mechanism

The URL shortening method is to run the URL through a HMAC sha1 hashing function with a random salt to increase chances of uniqueness.

Here we use HMAC-SHA1 because it is a one-way hashing function and does not allow the url to be decoded from the hash itself (you need to look it up in the Redis table)

$hash = hmac_sha1($random_salt , $my_url);

Core Service Components

1. The API Servers

• The two API servers are implemented as HTTP Servers in Golang or Python
• These handle PUT and GET requests separately
• The services are running in docker containers with separate pods
• An API server receives the PUT request to encode a URL and looks up the URL
• If URL is found in the Redis table, the API returns: "Duplicate" in response
• If URL is not in the Redis table, it encodes it using a simple shortening algorithm
• returns HTTP 200 OK with the short URL only on successful write to the Redis table
• An API server receives a GET request including the shortened URL and does a lookup
• if the short url is found in the table, the original URL is returned using a query
• This URL is formatted into a 302 redirect response by the API HTTP Service