Storing Images and Running Containers

Storing Images

Docker Hub :

Docker Hub is a service provided by Docker for finding and sharing container images.

Visit https://hub.docker.com and create an account to store your images.

You can create any number of public repositories. Docker providers only one private repository for free. If you want more no.of private repositories, you can upgrade the plan.

We can push the image to the docker hub with “docker push” command. Before that, we need to do two things:

  1. Login to the docker hub using “docker login” command and provide username and password. Once successfully authenticated, you don’t need to login every time until you logged out using “docker logout” command.
  2. Tag the image with repository name and tag (or version number) using docker tag command.
$ docker login
Username:
Password:
Login Succeeded
$

 

$ docker images
REPOSITORY TAG    IMAGE ID     CREATED       SIZE
mywebsite  latest 44870583de0c 2 minutes ago 380MB
centos     7      1e1148e4cc2c 2 months ago  202MB
$
$ docker tag 44870583de0c learninghub/website:1.0
$ docker push learninghub/website:1.0
The push refers to repository [docker.io/learninghub/website]
bc6f85aad9de: Pushed
1d1fc5de57b2: Pushed
071d8bd76517: Mounted from library/centos
1.0: digest: sha256:4ff5eef21cc80f609d20064c62cf2f9caa3c27a363324732c9d83442b141beb3 size: 948
$

 

We can also store the images in the private repositories offered by different cloud providers.

  • Elastic Container Registry by AWS
  • Azure Container Registry by Microsoft Azure
  • Google Container Registry by Google
  • Docker Trusted Registry by Docker
  • Docker Registry by Quay.io 
  • And more…..

We can run our own local registry using docker “registry” image. 

$ docker run -d -p 5000:5000 
--restart always 
--name registry 
registry:2

To Store images in the local registry, tag the image with localhost:5000.

$ docker tag 44870583de0c localhost:5000/website:1.0
$ docker push localhost:5000/website:1.0

Run Containers

Run containers using the docker images with “docker run” command.

Syntax : docker run [options] image:tag

$ docker run -d 
--name mywebsite 
-p 8080:80 
mywebsite:latest
1c63111fed79a373b4cdc2ede2bf72f33139e0d90e124eec999790b47c65af8d
$

Here, -d for run in background ( as daemon), -p for to map host to container port (HostPort:ContainerPort).

To check the running containers, use “docker ps” commands.

$ docker ps
CONTAINER ID IMAGE            COMMAND                CREATED       STATUS       PORTS                NAMES
1c63111fed79 mywebsite:latest "/usr/sbin/httpd -D …" 4 minutes ago Up 4 minutes 0.0.0.0:8080->80/tcp mywebsite

Now, access the website using http://HOST-IP:8080

To run the container use, “kubectl run” command or create a yaml.

$ kubectl run  mywebsite --image=learninghub/website:1.0
deployment.apps "mywebsite" created

To check the pods, use kubectl get pods 

$ kubectl get pods
NAME                       READY STATUS  RESTARTS AGE
mywebsite-5c588f6774-zmlzp 1/1   Running 0        54s

To Access the website, we have to create a service for this deployment, using “kubectl expose” command.

$ kubectl expose deployment mywebsite --port=80 --type=NodePort
service "mywebsite" exposed

You can access the website using worker node IP address.

If you have kubernetes cluster in the cloud, you can create a service with load balancer as well.

AWS Shared Responsibility

 

Security and Compliance is a shared responsibility between AWS and the customer.

AWS is responsible for securing the underlying infrastructure that runs all of the services offered in the AWS Cloud.

The customer is responsible for anything that you on the cloud or connects to the cloud.

AWS is responsible for Security of the Cloud :

  • Protecting the global infrastructure.
  • Protection from external attacks of the physical AWS services and resources.
  • Security configurations of managed services like Amazon DynamoDB, RDS, RedShift, EMR, and other services.
  • Operating, managing and controlling the components from the host operating system and virtualization layer.

The customer is responsible for Security in the Cloud :

  • Authentication, authorization, integrity, and encryption of the client-side data
  • The encryption of server-side information via file system or directly into the data storage
  • Securing the configuration related to the network and networking devices configuration like firewall, using securing controls like NACLs (network access control lists) and security groups.
  • Deploying, configuring and deploying security baselines within their AWS available services.

shared_resp.jpg

Visit: Shared Responsibility Model – AWS

Cloud Terminology

# High availability

High availability refers to a system or component that is durable and operates continuously without failure for a higher than normal period. Availability is usually expressed as a percentage of uptime in a given year. High Availability relative to “100% operational” or “never failing.”

 

# Fault tolerant

Fault tolerance is the capability of a component or a computer system to respond to an unexpected software or hardware failure to deliver uninterrupted service and to continue operating properly in the event of the failure of its components.

# Scalability

Scalability is the capability of a system, network, or process to handle an increasing amount of work (load), or its potential to be enlarged to accommodate that growth in load.

# Elasticity

Elasticity is the ability to adapt workload changes by provisioning and de-provisioning resources in an autonomic manner, such that at each point in time the available resources match the current demand as closely as possible.

AWS CLI Installation

The easiest way to install the AWS CLI is using pip, it is a package manager for Python that provides an easy way to install, upgrade, and remove Python packages and their dependencies.

Install Python pip

CentOS :
sudo yum update -y
sudo yum install -y python-pip
Ubuntu :
sudo apt-get update -y
sudo apt-get install -y python-pip

Install AWS CLI using pip

pip install awscli --upgrade --user

–upgrade option tells pip to upgrade any requirements that are already installed.
–user option tells pip to install the program to a subdirectory of your user directory to avoid modifying libraries used by your operating system.

Add the executable path to your PATH variable: ~/.local/bin
export PATH=$PATH:~/.local/bin
Load the profile into your current session
source ~/.bash_profile
Verify that the AWS CLI installed correctly by running aws --version.
aws --version
aws-cli/1.16.19 Python/2.7.5 Linux/3.10.0-862.11.6.el7.x86_64  botocore/1.12.9

Configure AWS CLI

To access aws services using cli, we need to provide the access key and secret key along with a default region. This can be done using a subcommand provided by aws cli.

$ aws configure
AWS Access Key ID [None]: XXXXXXXXXXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Default region name [None]: us-east-1
Default output format [None]: json

 I have used us-east-1 as default region and default output format is JSON. We can format the output as text and table formats as well.

aws configure command creates two configuration files located under .aws in user home directory.

ls ~/.aws/
config credentials
cat config 
[default]
output = json
region = us-east-1
cat credentials 
[default]
aws_access_key_id = XXXXXXXXXXXXXXXXXXXX
aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Verify if AWS CLI configuration

 Describe your regions
aws ec2 describe-regions
{
    "Regions": [
        {
            "Endpoint": "ec2.ap-south-1.amazonaws.com", 
            "RegionName": "ap-south-1"
        }, 
        {
            "Endpoint": "ec2.eu-west-3.amazonaws.com", 
            "RegionName": "eu-west-3"
        }, 
        {
            "Endpoint": "ec2.eu-west-2.amazonaws.com",
            "RegionName": "eu-west-2"
        }, 
        .....
        .....
        {   "Endpoint": "ec2.us-west-2.amazonaws.com", 
            "RegionName": "us-west-2" 
        } 
    ] 
}

Docker Installation

Docker is available in two editions:

  1. Community Edition (CE)
  2. Enterprise Edition (EE)

Docker Community Edition (CE) is ideal for individual developers and small teams looking to get started with Docker and experimenting with container-based apps.

Docker Enterprise Edition (EE) is designed for enterprise development and IT teams who build, ship, and run business critical applications in production at scale.

Let’s see how to install Docker CE on Linux.

Step 1 : Uninstall old versions

Older versions of Docker were called docker or docker-engine. If these are installed, uninstall them, along with associated dependencies.

CentOS :

$ sudo yum remove docker 
                  docker-client 
                  docker-client-latest 
                  docker-common 
                  docker-latest 
                  docker-latest-logrotate 
                  docker-logrotate 
                  docker-selinux 
                  docker-engine-selinux 
                  docker-engine

Ubuntu :

$ sudo apt-get remove docker docker-engine docker.io

Step 2 : Set up the Repository.

CentOS :

  1. Install required packages. yum-utils provides the yum-config-manager utility, and device-mapper-persistent-data and lvm2 are required by the devicemapper storage driver.
    $ sudo yum install -y yum-utils 
      device-mapper-persistent-data 
      lvm2
    
  2. Use the following command to set up the stable repository. You always need the stable repository.
    $ sudo yum-config-manager 
        --add-repo 
        https://download.docker.com/linux/centos/docker-ce.repo

Ubuntu :

  1. Update the apt package index:
    $ sudo apt-get update
    
  2. Install packages to allow apt to use a repository over HTTPS:
    $ sudo apt-get install 
        apt-transport-https 
        ca-certificates 
        curl 
        software-properties-common
    
  3. Add Docker’s official GPG key:
    $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -Use the following command to set up the stable repository.
  4. Use the following command to set up the stable repository.
    $ sudo add-apt-repository 
       "deb [arch=amd64] https://download.docker.com/linux/ubuntu 
       $(lsb_release -cs) 
       stable"

Step 3 : Install Docker CE

CentOS :

$ sudo yum install docker-ce

Ubuntu :

  1. Update the apt package index.
    $ sudo apt-get update
    
  2. Install the latest version of Docker CE.
    $ sudo apt-get install docker-ce

Step 4 : Start Docker

$ sudo systemctl start docker
$ sudo systemctl enable docker

Step 5 : Verify Docker

  1. Verify that Docker CE is installed correctly by running the hello-world image.
$ sudo docker run hello-world

Uninstalling Docker CE

Ubuntu :

  1. Uninstall the Docker CE package:
    $ sudo apt-get purge docker-ce
    
  2. Images, containers, volumes, or customized configuration files on your host are not automatically removed. To delete all images, containers, and volumes:
    $ sudo rm -rf /var/lib/docker

CentOS:

  1. Uninstall the Docker EE package:
    $ sudo yum -y remove docker-ce
    
  2. Delete all images, containers, and volumes (because these are not automatically removed from your host):
    $ sudo rm -rf /var/lib/docker
    
  3. If desired, remove the devicemapper thin pool and reformat the block devices that were part of it.

Installation Script

The easiest way to install docker with installation script. Below is the script url. Make sure the user should have sudo privileges.

https://get.docker.com/

Execute the below two commands to install docker.

$ curl -fsSL get.docker.com -o get-docker.sh
$ sh get-docker.sh

$ curl -fsSL get.docker.com -o get-docker.sh
$ sh get-docker.sh
# Executing docker install script, commit: fc04d2c
+ sudo -E sh -c 'yum install -y -q yum-utils'
Package yum-utils-1.1.31-42.el7.noarch already installed and latest version
+ sudo -E sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
+ '[' edge '!=' stable ']'
+ sudo -E sh -c 'yum-config-manager --enable docker-ce-edge'
Loaded plugins: fastestmirror
............................................
...............................................
...................................................

Start docker 

$ sudo systemctl start docker

Check version of Docker

$ docker --version 
Docker version 18.02.0-ce, build fc4de44

After installation completed, execute below command to run docker commands as non root.

$ sudo usermod -aG docker centos

Docker CE Installation on Red Hat 7

Docker CE is not officially supported on RHEL but you can get the latest Docker CE installed on it.

Here are the steps.

Step 1 : Install yum-utils and epel-release.

yum-utils

# yum install -y yum-utils

epel-release

# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
--2018-03-24 10:28:49-- http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Resolving dl.fedoraproject.org (dl.fedoraproject.org)... 209.132.181.25, 209.132.181.23, 209.132.181.24
Connecting to dl.fedoraproject.org (dl.fedoraproject.org)|209.132.181.25|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15080 (15K) [application/x-rpm]
Saving to: ‘epel-release-latest-7.noarch.rpm’

100%[==========================================================================================================================>] 15,080 57.0KB/s in 0.3s

2018-03-24 10:28:50 (57.0 KB/s) - ‘epel-release-latest-7.noarch.rpm’ saved [15080/15080]

#
# rpm -ivh epel-release-latest-7.noarch.rpm

Step 2 : Add Docker CE to yum repos.

# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Before docker-ce installation, we should install container-selinux. If not docker-ce will fail due to container-selinux dependencies. 

Also, if epel-release package not installed, then also docker-ce will fail with below error.

Error: Package: docker-ce-18.03.0.ce-1.el7.centos.x86_64 (docker-ce-stable) Requires: pigz

Step 3: Install container-selinux package.

# yum install -y http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.33-1.git86f33cd.el7.noarch.rpm

Step 4: Install Docker CE

# yum install -y docker-ce
# docker --version
Docker version 18.03.0-ce, build 0520e24
#

Step 5: Restart docker service and enable it.

# systemctl restart docker
# systemctl enable docker 
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.

Step 5: Now run nginx container to test.

# docker run --name webserver -d -p 9090:80 nginx

Now check in web browser with below url

http://your-IP-address:9090/

nginx.PNG

How to access EC2 instance without .pem file.

Here are the steps to login into Ec2 instance without .pem file.

Login into Ec2 instance with .pem file.

$ ssh -i "aws-key.pem" ec2-user@ec2-35-154-198-16.ap-south-1.compute.amazonaws.com

Create a new user to access the Ec2 instance with strong password

$ sudo useradd USER_NAME
$ passwd USER_NAME
Changing password for user root.
New password: 
BAD PASSWORD: The password fails the dictionary check - it is too simplistic/systematic
Retype new password: 
passwd: all authentication tokens updated successfully.
$

Add the user to sudoers file visudo command and add the below line

USER_NAME ALL=(ALL) ALL

$ sudo visudo

Enable password authentication by editing /etc/ssh/sshd_config file.

Initial Configuration:

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication no

Change to below configuration:

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication yes
#PermitEmptyPasswords no
#PasswordAuthentication no

Comment for no and uncomment for yes

Restart sshd service.

# service sshd restart
Redirecting to /bin/systemctl restart sshd.service
#

Now, logout from the instance and login with your password.

$ ssh USER_NAME@ec2-35-154-198-16.ap-south-1.compute.amazonaws.com

It will ask for your password. Enter the password and hit Enter key.

Docker Installation on CentOS 7 with Installation Script

The easiest way to install docker with installation script. Below is the script url. Make sure the user should have sudo privileges.

https://get.docker.com/

Execute the below two commands to install docker.

$ curl -fsSL get.docker.com -o get-docker.sh
$ sh get-docker.sh

[centos@docker-server ~]$ curl -fsSL get.docker.com -o get-docker.sh
[centos@docker-server ~]$ sh get-docker.sh
# Executing docker install script, commit: fc04d2c
+ sudo -E sh -c 'yum install -y -q yum-utils'
Package yum-utils-1.1.31-42.el7.noarch already installed and latest version
+ sudo -E sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
+ '[' edge '!=' stable ']'
+ sudo -E sh -c 'yum-config-manager --enable docker-ce-edge'
Loaded plugins: fastestmirror
............................................
...............................................
...................................................

After completed, execute below command to run docker commands as non root.

sudo usermod -aG docker your-username

[centos@docker-server ~]$ sudo usermod -aG docker centos

Start docker 

[centos@docker-server ~]$ sudo systemctl start docker 

Check version of Docker

[centos@docker-server ~]$ docker --version 
Docker version 18.02.0-ce, build fc4de44
[centos@docker-server ~]$ 

To uninstall docker, execute below command

[centos@docker-server ~]$ sudo yum remove docker-ce

 

Ansible Installation on RHEL 7

The easiest way to install Ansible is by adding EPEL (Extra Packages for Enterprise Linux) repository.

Use below command to add epel repository.

[root@ansible-server ~]# rpm -ivh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Retrieving http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
warning: /var/tmp/rpm-tmp.C01zA7: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Preparing... ################################# [100%]
Updating / installing...
 1:epel-release-7-11 ################################# [100%]
[root@ansible-server ~]# 

Verify the epel repository

[root@ansible-server ~]# yum repolist |grep epel
epel/x86_64 Extra Packages for Enter 12,353
[root@ansible-server ~]#

After added epel repository, we can install Ansible using following command

[root@ansible-server ~]# yum install ansible -y

After installed successfully, we can verify the version by executing below command

[root@ansible-server ~]# ansible --version
ansible 2.4.2.0
 config file = /etc/ansible/ansible.cfg
 configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
 ansible python module location = /usr/lib/python2.7/site-packages/ansible
 executable location = /bin/ansible
 python version = 2.7.5 (default, May 3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
[root@ansible-server ~]#

 

Learn Linux : V 2.6 Released

An android Linux Book

You can learn Linux  freely from this app up to what you need.

What’s new in this release?
  • Minor bug fixes.
  • New API integrated.
  • Added terminal block to commands.
  • Redesigned chapters with borders.
  • Corrections in chapters.

Google