{"id":345,"date":"2019-04-12T21:08:36","date_gmt":"2019-04-12T21:08:36","guid":{"rendered":"https:\/\/bootstrap-it.com\/blog\/?p=345"},"modified":"2019-04-12T21:08:36","modified_gmt":"2019-04-12T21:08:36","slug":"an-in-depth-introduction-to-docker-on-aws","status":"publish","type":"post","link":"https:\/\/bootstrap-it.com\/blog\/?p=345","title":{"rendered":"An in-depth introduction to Docker on AWS"},"content":{"rendered":"
<\/a><\/a><\/a><\/a><\/a><\/div>\n
\"\"\/
Photo by frank mckenna<\/a> on Unsplash<\/a><\/figcaption><\/figure>\n\n\n\n

Container virtualization\u200a\u2014\u200amost visibly represented by Docker\u200a\u2014\u200ais a
server paradigm that will likely drive enterprise computing for years
to come.<\/p>\n\n\n\n

The Cloud is the most obvious and logical platform for container
deployment.<\/p>\n\n\n\n

Amazon Web Services largely dominates the cloud computing world.
Add it up. If you\u2019re interested in getting a piece of all this action, you\u2019ll
definitely want to figure out how it all works.<\/p>\n\n\n\n

First, though, let\u2019s quickly define some key terms.<\/p>\n\n\n\n

Virtualization<\/h3>\n\n\n\n

Virtualization is the division of physical computer and networking resources into smaller, more flexible units, presenting these smaller units to users as though each was a discrete resource.<\/p>\n\n\n\n

The idea is that, instead of assigning specific computing tasks to individual physical servers\u200a\u2014\u200awhich may sometimes end up being over- or underused
\u200a\u2014\u200aa single physical server can be logically divided into as few or as many virtual servers as needed.<\/p>\n\n\n\n

That means, as the figure below illustrates, there can be dozens of individually installed operating systems (OS) running side by side on the same hard drive. Each OS is effectively unaware that it isn\u2019t all alone in its local environment.<\/p>\n\n\n\n

\"\"\/
Multiple applications being served through physical servers or, via VMs, from a single virtualization server<\/figcaption><\/figure>\n\n\n\n

Practically, each operating system instance can be accessed remotely by both administrators and customers in exactly the same way as any other server.<\/p>\n\n\n\n

In this kind of environment, as soon as your virtual server completes its task or becomes unnecessary, you can instantly delete it. This will free up the resources it was using for the next task in the queue.<\/p>\n\n\n\n

There\u2019s no need to over-provision virtual servers to anticipate possible future needs, because future needs can be easily met whenever they arrive.<\/p>\n\n\n\n

In fact, today\u2019s virtual server might only live a few minutes or even seconds before, having completed its task, being shut down for good to make room for whatever\u2019s next. All this allows for far more efficient use of expensive hardware. It provides the ability to provision and launch new servers at will, either to test new configurations or add fresh power to your production services.<\/p>\n\n\n\n

Cloud computing providers like AWS use virtualized computers of one kind or another. The hundreds of thousands of Amazon EC2<\/a> instances, for example, all run on top of the open source Xen<\/a> or KVM<\/a> hypervisors<\/a>\u200a\u2014\u200awhich are themselves installed and running on many thousands of physical servers maintained in Amazon\u2019s vast server farms.<\/p>\n\n\n\n

Whatever hypervisor technology is being used, the goal is to provide a largely automated hosting environment for multiple complete, self-contained virtual computers.<\/p>\n\n\n\n

Containers like Docker, on the other hand, aren\u2019t standalone virtual machines but are modified file systems sharing the operating system kernel of their physical host. That\u2019s what we\u2019ll discuss next.<\/p>\n\n\n\n

Containers<\/h3>\n\n\n\n

What are containers? Well, for one thing, they\u2019re not hypervisors. Instead, they\u2019re extremely lightweight virtual servers that, as you can see from the figure, rather than running as full operating systems, share the underlying kernel of their host OS.<\/p>\n\n\n\n

\"\"\/
Virtualized containers running with access to their host\u2019s kernel and hardware resources<\/figcaption><\/figure>\n\n\n\n

Containers can be built from plain-text scripts, created and launched in seconds, and easily and reliably shared across networks. Container technologies include the Linux Container<\/a> project, which was Docker\u2019s original inspiration.<\/p>\n\n\n\n

The script-friendly container design makes it easy to automate and remotely manage complex clusters of containers, often deployed as microservices.<\/p>\n\n\n\n

Microservices is a compute services architecture where multiple containers are deployed, each with a distinct yet complementary role. You might, therefore, launch one container as a database back-end, another as a file server, and a third as a web server.<\/p>\n\n\n\n

Docker<\/h4>\n\n\n\n

As I explore in one or two of my Pluralsight courses<\/a>, a Docker container is an image whose behavior is defined by a script. The container is launched as a software process that\u2019s cunningly disguised as a server.<\/p>\n\n\n\n

But what\u2019s an image? It\u2019s a software file containing a snapshot of a full operating system file system. Everything necessary to launch a viable virtual server is included.<\/p>\n\n\n\n

An image might consist of just a base operating system like Ubuntu Linux, or the tiny and super-fast Alpine Linux. But an image could also include additional layers with software applications like web servers and databases. No matter how many layers an image has and how complicated the relationships between them might be, the image itself never changes.<\/p>\n\n\n\n

When, as shown in the next figure, an image is launched as a container, an extra writable layer is automatically added into which the record of any ongoing system activity is saved.<\/p>\n\n\n\n

\"\"\/
A simple MySQL\/Ubuntu Docker image illustrated with its writable data layer<\/figcaption><\/figure>\n\n\n\n

What do folks commonly do with their Docker containers? Often, they\u2019ll load up some kind of app development project to test how it will work, and then share it with team members for feedback and updates. When the app is complete, it can be launched as a cluster of containers (or \u201cswarm\u201d as Docker calls it) that can be programmatically and instantly scaled up or down according to user demand.<\/p>\n\n\n\n

While Docker is a Linux-based technology and requires a Linux kernel to run, running remote or even local Docker containers on Mac or Windows machines is possible through either the Docker for Mac or Docker for Windows apps or for older machines, through the Docker Machine tool.<\/p>\n\n\n\n

Cloud computing<\/h3>\n\n\n\n

Cloud computing is the provision of on-demand, self-service compute,
memory, and storage resources remotely over a network.<\/p>\n\n\n\n

Since cloud-based services are billed in very small increments, you can
quickly configure and launch a wide range of projects. And since the
resources are all virtual, launching them as part of an experiment or to
solve some short-term problem will often make lots of sense. When the
work\u2019s all done, the resource is shut down.<\/p>\n\n\n\n

Cloud platforms let you do things that would be impossible\u200a\u2014\u200aor impossibly
expensive\u200a\u2014\u200aanywhere else.<\/p>\n\n\n\n

Unsure how long your project will run or how much demand it will attract?
Perhaps purchasing, building, and housing all the expensive hardware you\u2019d need to properly support your project in-house can\u2019t be justified.<\/p>\n\n\n\n

Investing heavily in server, cooling, and routing equipment might just not
make sense.<\/p>\n\n\n\n

But if you could rent just enough of someone else\u2019s equipment to match
fast-changing demand levels and pay only for what you actually use, then
it might work.<\/p>\n\n\n\n

AWS<\/h3>\n\n\n\n

There\u2019s no shortage of ways to manage Docker containers on AWS. In fact, between frameworks, orchestration interfaces, image repositories,
and hybrid solutions, the variety can get confusing.<\/p>\n\n\n\n

This article won\u2019t dive deeply into every option, but you should at least be aware of all your choices:<\/p>\n\n\n\n

Amazon\u2019s EC2 Container Service<\/a>(ECS) leverages specially configured EC2 instances as hosts for integrated Docker containers. You don\u2019t have to get your hands dirty on the EC2 instance itself, as you can provision and administrate your containers through the ECS framework. ECS now offers greater abstraction (and simplicity) through their Fargate mode option.<\/p>\n\n\n\n

AWS CloudFormation<\/a> allows you to configure any combination of AWS resources into a template that can be deployed one or many times. You can include specified dependencies and custom parameters in the template. Given its self-contained and scriptable design, CloudFormation is a natural environment for Docker deployments. In fact, Docker itself offers its Docker for AWS service (currently in beta), that will automatically generate a CloudFormation template to orchestrate a swarm of Docker containers to run on AWS infrastructure within your account.<\/p>\n\n\n\n

AWS Elastic Beanstalk<\/a> effectively sits on top of ECS. It allows you to deploy your application across all the AWS resources normally used by ECS, but with virtually all of the logistics neatly abstracted away. Effectively, all you need in order to launch a fully scalable, complex microservices environment is a declarative JSON-formatted script in a file called Dockerrun.aws.json<\/code>. You can either upload your script to the GUI or, from an initialized local directory using the AWS Beanstalk CLI.<\/p>\n\n\n\n

Amazon Elastic Container Service for Kubernetes<\/a> (EKS) is currently still in preview. It\u2019s a tool allowing you to manage containers using the open source Kubernetes orchestrator, but without having to install your own clusters. Like ECS, EKS will deploy all the necessary AWS infrastructure for your clusters without manual intervention.<\/p>\n\n\n\n

Docker for AWS<\/a> is, at the time of writing, still in beta. Using its browser interface, you can use the service to install and run a \u201cswarm of Docker Engines\u201d that are fully integrated with AWS infrastructure services like auto scaling, load balancing (ELB), and block storage.<\/p>\n\n\n\n

Docker Datacenter (now marketed as part of Docker Enterprise Edition<\/a>) is a joint AWS\/Docker project that provides commercial customers with a more customizable interface for integrating Docker with AWS, Azure, and IBM infrastructures.<\/p>\n\n\n\n

Docker Cloud<\/a>, much like Docker Datacenter, offers a GUI, browser-based console for managing all aspects of your Docker deployments. This includes administration for your host nodes running in public clouds. The big difference is that, unlike Datacenter, the Docker Cloud administration service is hosted from its own site. There\u2019s no server software to install on your own equipment.<\/p>\n\n\n\n

Docker Hub<\/a> is probably the obvious first place to look for and to share Docker images. Provided by Docker itself, Docker Hub holds a vast collection of images that come pre-loaded to support all kinds of application projects. You can find and research images on the hub.docker.com web site, and then pull them directly into your own Docker Engine environment.<\/p>\n\n\n\n

EC2 Container Registry<\/a>(ECR) is Amazon\u2019s own image registry to go with their EC2 Container Service platform. Images can be pushed, pulled, and managed through the AWS GUI or CLI tool. Permissions policies can closely control image access only to the people you select.<\/p>\n\n\n\n

I think you\u2019re ready to start. If you haven\u2019t yet, do head over to the
Amazon Web Services site to create an AWS account. In case you\u2019re not
yet familiar with how this all works, new accounts get a generous full year
of experimentation with any service level that\u2019s eligible for the Free Tier.
Assuming you\u2019re still in your first year, nothing we\u2019re going to do in this
course should cost you a penny.<\/p>\n\n\n\n

Next, we\u2019ll pop the lid off Docker and see how it works at its most basic level: your laptop command line. Technically, this has very little relevance to AWS workloads, but it\u2019ll be a great way to better understand the workflow.<\/p>\n\n\n\n

Introduction to Docker<\/h3>\n\n\n\n

Properly visualizing how all the many AWS parts work will probably be
easier if you first understand what\u2019s going on under the hood with Docker
itself. So in this article I\u2019ll walk you through launching and configuring
a simple Docker container on my local workstation.<\/p>\n\n\n\n

Ready to go?<\/p>\n\n\n\n

The Docker command line<\/h4>\n\n\n\n

Let\u2019s see how this thing actually works. I\u2019m going to get Docker up and running on my local workstation and then test it out with a quick hello-world operation. I will then pull a real working Ubuntu image and run it.<\/p>\n\n\n\n

I won\u2019t go through the process of installing Docker on your machine here for a few reasons. First of all, the specifics will vary greatly depending on the operating system you\u2019re running. But they\u2019re also likely to frequently change, so anything I write here will probably be obsolete within a short while. And finally, none of this is all that relevant to AWS. Check out Docker\u2019s own instructions at docs.docker.com\/install<\/a>.<\/p>\n\n\n\n

Along the way I\u2019ll try out some of Docker\u2019s command line tools, including creating a new network interface and associating a container with it. This is the kind of environment configuration that can be very useful for real-world deployments involving multiple tiers of resources that need to be logically separated.<\/p>\n\n\n\n

Most Linux distributions now use systemd<\/a> via the systemctl<\/a> command to handle processes. In this case systemctl start docker<\/code> will launch the Docker daemon if it\u2019s not already running. systemctl status docker<\/code> will return some useful information, including in-depth error messages if something has gone wrong. In this case, everything looks healthy.<\/p>\n\n\n\n

# systemctl start docker
# systemctl status docker<\/pre>\n\n\n\n
That\u2019s the only Linux-specific bit. From here on in we\u2019ll be using commands that\u2019ll work anywhere Docker\u2019s properly installed.<\/p>\n\n\n\n
Launch a container<\/h4>\n\n\n\n
Running commands from the Docker command line always begins with the word \u201cdocker\u201d. The normal first test of a newly installed system is to
use docker run<\/code> to launch a small image\u200a\u2014\u200athe purpose-built \u201chello-world\u201d image in this case.<\/p>\n\n\n\n
As you can tell from the output below, Docker first looked for the image on the local system. Docker is particularly efficient in that way. It will always try to reuse locally available elements before turning to remote sources.<\/p>\n\n\n\n
In this case, since there are no existing images in this new environment, Docker goes out to pull hello-world from the official Docker library.<\/p>\n\n\n\n
$ docker run hello-world
Unable to find image \u2018hello-world:latest\u2019 locally
latest: Pulling from library\/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:66ef312bbac49c39a89aa9bcc3cb4f3c9e7de3788c9
44158df3ee0176d32b751
Status: Downloaded newer image for hello-world:latest2.1. 
Hello from Docker!
This message shows that your installation appears to be
working correctly. To generate this message, Docker took the
following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the \u201chello-world\u201d image
from the Docker Hub. (amd64)
3. The Docker daemon created a new container from that
image which runs the executable that produces the output you
are currently reading.
4. The Docker daemon streamed that output to the Docker client,
which sent it to your terminal.
To try something more ambitious, you can run an Ubuntu container
with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https:\/\/cloud.docker.com\/<\/a>
For more examples and ideas, visit:
https:\/\/docs.docker.com\/engine\/userguide\/<\/a><\/pre>\n\n\n\n
The full output of this command includes a useful four part description of what just happened. The Docker client contacted the Docker daemon which proceeded to download the hello-world image from the repository. The image is converted to a running container by the docker run command whose output is streamed to our command line shell\u200a\u2014\u200athe Docker client.<\/p>\n\n\n\n
Let me break that jargon down for you just a bit:<\/p>\n\n\n\n