Learning Microservices Architecture with Bluemix and Docker (Part 1)

Share: Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInShare on RedditEmail this to someonePrint this page

MicroserviceBanner1

The Microservices architecture paradigm has become one of the hottest topics in web development, as web service giants such as Netflix and Spotify have adopted the architecture. Service based application design is not a new idea but the rise of Docker and containers in general have been driving factors. For those who are unfamiliar with Containers, the Bluemix docs give one of the best descriptions.

Containers are virtual software objects that include all the elements that an application needs to run. A container is built from an image, which holds an application. Each image includes just the app and its dependencies, running as an isolated process on the host operating system. Therefore, it has the benefits of resource isolation and allocation, but is more portable and efficient. Containers help you build high-quality apps, fast.

Docker is an open source project that has recently become the most popular and widely used container implementation. They provide lightweight containers that allow developers to deploy applications or APIs into containers and spin up instances rapidly. Docker’s use of reusable container images makes container replication easy. Swift and simple container replication makes Docker ideal for scaling up the number of containers to handle increased load. Docker’s strengths make it ideal for implementing Microservices Architecture applications.

Aside from the benefits provided by Docker containers, the Microservices design paradigm provides a lot of additional benefits:

  • Ability to scale up parts of an application as needed, instead of scaling everything.
  • If one service crashes it should not take down the entire application.
  • Allows for continuous delivery, as individual services can be updated with zero downtime.
  • Decrease overall app complexity by breaking monolithic app into simple modular services.
  • Easily replace or repair old services. (Avoids developing legacy systems)
  • Allows the use of different languages, hardware and databases. (use the right tool for the job)

There are a lot of other blogs and articles talking about the advantages and disadvantages. Instead, this article will assume you know what Microservices Architecture is and want to dive into it. If you are looking to learn more about Microservices Architecture read these great posts and come back and try this tutorial.

To give you a feel for the Microservice development process we will be building a simple online store written using the MEAN stack (MongoDB, Express, Angular, and Node.js). You will not need to be an expert in the MEAN stack to complete the tutorial. Basic understanding of Javascript and terminal commands will suffice.

The Front end will be given to you. We will instead focus on re-engineering the backend (where the Microservices exist).

Screen Shot 2015-07-17 at 4.20.59 PM

Plan of attack

  1. Deploy to Bluemix using the deploy to bluemix button
  2. Download project repo
  3. Run project locally
    • Get MongoLab connection data from bluemix
    • Export Connection URI to environment variables
  4. Play around with the application, kick the tires.
  5. Learn how it works internally
  6. Start Breaking the monolithic application apart (Part 2)
    • create service directories and files
    • set up service dependencies
    • Move app.js APIs over to the services
    • Test the services with Postman.
  7. Dockerize the services (Part 3)
    • Create the Dockerfiles
    • Create images from the Dockerfiles
  8. Run the Containers
    • Test the running containers again
  9. Tie the services into the front end
    • Clean up app.js to only serve the frontend
    • Update index.html with the microservice endpoints
  10. Dockerize the front end
    • Create front end image from Dockerfile
    • run front end client with docker
  11. Play with your new Microservices Architecture application

We will start with a single monolithic backend server that handles a handful of APIs that are needed to run the store’s front end. We will then break this back end up into three distinct services that will each run in separate Docker containers. The three services are:

  1. the Product API – serves up json containing data for all the products in the store.
  2. the Cart API – Handles all cart functionality, including loading cart contents and verifying payments.
  3. the Review API – Stores and serves up the reviews for all store products.

Let us begin by hosting the entire application on Bluemix as a Cloud Foundry Application. After that we will take the app down, break it apart, Dockerize the pieces and finally, plug everything back together.

Getting Started

(note: this tutorial assumes you have node.js, Docker and npm installed, get that done first)

Let’s deploy the app to get things rolling. Hit the “Deploy to Bluemix button” below.

button

The deploy to bluemix button will clone the repo, install dependencies and bind the MongoLab Database that we will use during development. Let the deploy process complete and after a few minutes you will have a fully functional store application. The bluemix console should give you a link to your newly created application. When you first open the store you’ll notice that it is completely empty! Lets add some fake data to our products database.

This isn’t a tutorial about how to fake json data, so we’ve done the heavy lifting for you with a handy json faker built right into the app! you are welcome. To use the faker API go to “<unique appname>.mybluemix.net/faker/15″ this will hit the faker api on the server and tell it to clear the database then create 15 fake products (or create any number you want). After hitting that link, you can go back to the main page (refresh) and see all your generated products. Now that you can play around with the app, go ahead and add products to the cart, try checking out from the cart page (click the cart link).

Screen Shot 2015-07-13 at 9.28.11 AM

Note: At this point you may be wondering why the products in this store are all countries, the faking library didn’t offer an option for faking product names. I decided to use the location faker. You can think of the store as a timeshare or vacation destination store.

To checkout, all you need to do is type in a fake number (of any length) to represent your gift card number. The expiration month and year cannot be a past date, the server will warn you if you type in an expired date.

For now the application APIs are running in a single app.js file (look into the code and you can see all the APIs). Lets dig in and learn how the back end works internally.

Clone or download the project source from this repo.

How the store works

monolithic

The Store Front

The application loads the Store front by accessing the Products API on the backend. The returned JSON comes from the Product collection in the MongoDB and shows up as products on the front end. Opening up the Reviews tab on a product reveals the product’s reviews. You will also notice that the Nav Bar shows the number of items in your cart. This is done by querying the Cart API to get the number of Items in the cart when the page loads. Once the page is loaded, adding and removing items from the cart will update a variable in the front end controllers.

Reviews

You can write a new review and save it. The front end will send the form data to the Review API on the backend and save the review by appending it to the specified product in the Products Collection in the Database.

Cart Page

Clicking on the cart logo in the Nav Bar takes us into the the Cart, on load Product JSON data is returned by the cart API on the backend. This works by searching the MongoDB Products collection for products that have the “inCart” attribute set to true. The cart JSON is then sent to the cart page and loaded into a table. The “inCart” variable is updated by the product API when an item is moved into or out of the cart. The checkout feature is also handled by the Cart API. You will type in a fake card number along with an expiration date. The API will then tell you if the card is Valid (unexpired) to complete the order.

Data Faker

The faker API is not tied to the front end in any way. You access this API to create fake data by hitting the “/faker/:numberOfProducts” endpoint. The API clears out all old data collections and replaces the documents with JSON created using the faker.js node_module.

The Monolith

In the above diagram you can see that all the APIs and Models are all encapsulated and Screen Shot 2015-07-16 at 11.57.10 AMrunning on the same server. The models are Mongoose schema definitions that allow us to write and read store objects to the MongoDB. These models are stored in the /models/ directory and exported as modules that are then used in the app.js file.

app.js is where the core of the application exists. it contains all the APIs and routes for serving up the front end. Here is a detailed overview of its contents:

  • DB setup – loads our mongoose models and connects them to the MONGO_URI that is parsed out of environment variables.
  • Express setup – loads express dependencies, adds api router and api logging through middleware.
  • Cart API – Returns the cart contents, the number of items in the cart, and manages the cart checkout process payment validation.
  • Reviews API – Responsible for serving reviews and adding newly created reviews to the store.
  • Products API – Serves all the product data required by the front end client. Also updates product’s inCart attribute when added and removed from cart.
  • Faker API – clears database and fakes product data for testing.

API technical details

This section is to be used as reference for testing and if you want to learn more about all the APIs that the backend offers.

Product API Endpoints

The Product API is used by the front end to get Product data in JSON form. The JSON is then used to populate the store front. The product reviews are managed by the review API but are saved in the Product collection. This means all reviews are loaded through the products API. The product API is also responsible for setting a product’s “inCart” attribute in the DB. this is what the Cart API uses to determine which products are in the cart.

  1. /api/products
    1. GET: returns all products as JSON
  2. /api/products/:product_id
    1. GET: returns the product as JSON matching the given product_id
    2. PUT: Updates the specified product’s “inCart” attribute to equal the passed “inCart” value. The inCart value is passed in the PUT request’s JSON payload. example payload: “{“incart”:”true”}”

Cart API Endpoints

The Cart API accesses the mongo Product collection and counts the number of products that have the “inCart” attribute set to true. The count is used for updating the nav-bar cart count. This same method is used to load the products on the cart page. The Checkout component of this API is responsible for verifying whether or not the card used at checkout is expired or not.

  1. /api/cart

    1. GET: returns all products that are in the cart (JSON)
  2. /api/cart/count

    1. GET: returns count of items in the cart.
  3. /api/checkout/verifyPayment

    1. PUT: The card number, along with expiration year and month are passed to the endpoint as JSON, the API verifies that the card is not expired. API returns a status of “verified” or “expired” as JSON. example payload: “{“cardNumber”:””, “month”:”07″, “year”:”2015″ }”.

Review API Endpoints

This API Posts new reviews to the Product collection. Reviews are bound to specific products so a product_id is required as a url parameter. There is a review GET api endpoint that is not used in the application. I left it in for testing purposes.

  1. api/reviews/:product_id

    1. GET: returns the reviews for the specified product_id (JSON) [This API is no longer used by the application]
    2. POST: Given a valid JSON payload for a review the POST endpoint will add the review to the reviews collection. (JSON)

Faker API Endpoints

This tool generates fake reviews and appends them to fake products. the faked data is then written to the Product collection to be later consumed by the other APIs. Notice that this endpoint is not prefixed with “api/”.

  1. /faker/:count

    1. GET: the faker endpoint will delete all data in the reviews and product collections and create “count” number of fake products with two false reviews each and add them to the respective collections. A status is returned if data faking succeeds.

As previously stated our goal is to take the Product, review, and cart APIs out of app.js and move them into containers that will be our microservices. Once completed our app should look more like this:

Microservice

 

We will take app.js and break it up into three services. We will still connect to one database to reduce complexity. In a full size microservices architecture application it is acceptable and often recommended to use multiple databases. Our services will still use mongoose and the mongoose models to access the mongoDB, so we will need to copy these models into the individual services that require them. You can use the diagram above to see which models the APIs will require.

The next part of this three part series will outline the process of converting this Monolithic application into a Microservice Architecture application. Link to Part 2

Share: Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInShare on RedditEmail this to someonePrint this page
Miguel Clement

Miguel Clement

Miguel is a Computer Science Senior at Texas A&M Univeristy. He joined the jStart Emerging Technology Team in January 2015 and has been exploring the cutting edge ever since.
Miguel Clement
Miguel Clement

2 comments

Leave a Reply

Your email address will not be published. Required fields are marked *