Category Archives: Cloud Computing

How to whitelist Google IP address ranges in firewall using iptables

As an administrator, when you need to obtain a range of IP addresses for Google APIs and services’ default domains, you can refer to the following sources of information.

The default domains’ IP address ranges for Google APIs and services fit within the list of ranges between these 2 sources. (Subtract the usable ranges from the complete list.)

Once you get the IP address ranges, use the “`xargs“` command to update iptables.

google-ips-whitelist.sh

echo "8.8.4.0/24
8.8.8.0/24
8.34.208.0/20
8.35.192.0/20
23.236.48.0/20
23.251.128.0/19
34.64.0.0/10
34.128.0.0/10
35.184.0.0/13
35.192.0.0/14
35.196.0.0/15
35.198.0.0/16
35.199.0.0/17
35.199.128.0/18
35.200.0.0/13
35.208.0.0/12
35.224.0.0/12
35.240.0.0/13
64.15.112.0/20
64.233.160.0/19
66.102.0.0/20
66.249.64.0/19
70.32.128.0/19
72.14.192.0/18
74.114.24.0/21
74.125.0.0/16
104.154.0.0/15
104.196.0.0/14
104.237.160.0/19
107.167.160.0/19
107.178.192.0/18
108.59.80.0/20
108.170.192.0/18
108.177.0.0/17
130.211.0.0/16
136.112.0.0/12
142.250.0.0/15
146.148.0.0/17
162.216.148.0/22
162.222.176.0/21
172.110.32.0/21
172.217.0.0/16
172.253.0.0/16
173.194.0.0/16
173.255.112.0/20
192.158.28.0/22
192.178.0.0/15
193.186.4.0/24
199.36.154.0/23
199.36.156.0/24
199.192.112.0/22
199.223.232.0/21
207.223.160.0/20
208.65.152.0/22
208.68.108.0/22
208.81.188.0/22
208.117.224.0/19
209.85.128.0/17
216.58.192.0/19
216.73.80.0/20
216.239.32.0/19" | xargs -I% iptables -I INPUT -p tcp -s % -j ACCEPT

how to manage users with ansible

If you have multiple servers to manage, it can be a pain to manually add a new user, change a password, or lock an old account. Manually logging into all of your servers and performing these tasks is a real pain, and a huge waste of time.

Using ansible user module, you can manage users and ssh keys in a single run of playbook.

Create users

The home directory for the user will also be created by default. You have the option to choose your home directory by setting the home parameter.

Following playbook is for Red Hat/CentOS

You need to change user group for Debian based systems

authorize_users.yml

---
- hosts: tag_group_{{ env }}_webserver
  ignore_unreachable: true
  strategy: free
  gather_facts: False

  vars_files:
  - group_vars/all.yml

  vars:
    users:
      - tony
      - thor
      - hulk

  tasks:
  - name: Make sure we have a 'wheel' group
    group:
      name: wheel
      state: present
  - name: Allow 'wheel' group to have passwordless sudo
    lineinfile:
      dest: /etc/sudoers
      state: present
      regexp: '^%wheel'
      line: '%wheel ALL=(ALL) NOPASSWD: ALL'
      validate: visudo -cf %s
  - name: "Create user accounts and add users to groups"
    user:
      name: "{{ item }}"
      groups: "wheel"
      shell: /bin/bash
    loop: "{{ users }}"
  - name: Add sudoers users to wheel group
    user:
      name: "{{ item }}"
      groups: wheel
      append: yes
    loop: "{{ users }}"
  - name: "Add authorized keys"
    authorized_key:
      user: "{{ item }}"
      key: "{{ lookup('file', '~/.ssh/'+ item + '.pub') }}"
      state: present
    with_items: "{{ users }}"

Running:

“`$ ENV=prod; ansible-playbook   -i inventories/$ENV –extra-vars “env=$ENV” authorize_users.yml“`

Remove Users

Removing an existing user is easy. You just have to set the ‘state’ parameter to ‘absent’. It executes the ‘userdel’ command in the background.

deauthorize_users.yml

---
- hosts: tag_group_{{ env }}_webserver
  ignore_unreachable: true
  strategy: free
  gather_facts: False

  vars_files:
  - group_vars/all.yml

  vars:
    users:
      - frodo
      - sam
      - gollum

  tasks:
  - name: "Remove from authorized keys"
    authorized_key:
      user: "{{ item }}"
      key: "{{ lookup('file', '~/.ssh/lintel/'+ item + '.pub') }}"
      state: absent
    with_items: "{{ users }}"

  - name: "Remove from authorized keys from root"
    authorized_key:
      user: root
      key: "{{ lookup('file', '~/.ssh/lintel/'+ item + '.pub') }}"
      state: absent
    with_items: "{{ users }}"

  - name: Remove users
    user:
      name: "{{ item }}"
      remove: yes
      state: absent
    loop: "{{ users }}"

 

Running:

“`$ ENV=prod; ansible-playbook -i inventories/$ENV –extra-vars “env=$ENV” deauthorize_users.yml“`

aws

Howto list all instances in all regions from mutliple accounts using awscli – AWS

AWS Cloud spans 69 Availability Zones within 22 geographic regions around the world, with announced plans for 9 more Availability Zones and three more Regions in Cape Town, Jakarta, and Milan.

If you are using more than one region it takes much time to browse through all regions in a browser and check which instances are running.

To save time, we are using awscli command in a shell script which will list all instances from all regions. You can use multiple profile names.

scrot

 

You can specify multiple profile names as follows:

#!/bin/bash
for i in profile1 profile2
do
    OWNER_ID=`aws iam get-user --profile $i --output text | awk -F ':' '{print $5}'`
    tput setaf 2;echo "Profile : $i";tput sgr0
    tput setaf 2;echo "OwnerID : $OWNER_ID";tput sgr0
    for region in `aws --profile $i ec2  describe-regions --output text | cut -f4`
    do
        tput setaf 1;echo  "Listing Instances in region $region";tput sgr0
        aws ec2 describe-instances --query 'Reservations[*].Instances[*].[Tags[?Key==`Name`].Value , InstanceId]' --profile $i --region $region --output text
    done &
done
wait

This will run jobs in parallel and exit when all jobs are completed.

How to update Route53 records after EC2 instance restart

Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. If you are not using Elastic IPs for your EC2 instances, chances are stopping and starting the server will result in different IPs after the instance comes back online. If you have A records pointing to those IPs in Route53 you will need a way to update them. After the script is executed, it will automatically gather the new Public IP and update the DNS record for it in Route53.

Edit 1:

As I am getting many comments on hosted zone parsing error… I wanted to add this

Before running script please check your host name is set to fully qualified domain name (FQDN) with this command
“`hostname -f“`

or

“`hostname -d“`

or

“`hostnamectl“`

If you don’t want to set host name system wide, feel free to modify the script manually

#!/bin/bash
# Author: Akhil Jalagam
# Description: update route53 after ec2 instance restart

HOSTNAME=$(hostname -f)
PUBLIC_IP=$(curl ident.me)
HOSTED_ZONE=$(hostname -d | awk -F"." '{print $(NF-2)".")
ZONE_ID=$(aws route53 --output json list-hosted-zones | jq --arg hosted_zone $HOSTED_ZONE. '.HostedZones[]  | select(.Name == $hosted_zone) | .Id' | awk -F"/" '{print $3}' | tr -d "\"")
RECORD_TYPE=$(aws route53 --output json list-resource-record-sets --hosted-zone-id $ZONE_ID --query "ResourceRecordSets[?Name == '$HOSTNAME.']" | jq ".[].Type" | tr -d "\"")
RECORD_SET="/tmp/updateroute53.json"

if [ -e "$RECORD_SET" ]
then
  rm -f $RECORD_SET
fi

echo "Updating resource record set"
echo "
{
    \"Comment\": \"Update record to reflect new public IP address\",
    \"Changes\": [
        {
            \"Action\": \"UPSERT\",
            \"ResourceRecordSet\": {
                \"Name\": \"$HOSTNAME.\",
                \"Type\": \"$RECORD_TYPE\",
                \"TTL\": 300,
                \"ResourceRecords\": [
                    {
                        \"Value\": \"$PUBLIC_IP\"
                    }
                ]
            }
        }
    ]
}" | tee -a /tmp/updateroute53.json

CHANGE_ID=$(aws route53 --output json change-resource-record-sets --hosted-zone-id $ZONE_ID --change-batch file:///$RECORD_SET | jq ".ChangeInfo.Id" | awk -F"/" '{print $3}' | tr -d "\"")
CHANGE_STATUS=$(aws route53 --output json get-change --id $CHANGE_ID | jq ".ChangeInfo.Status" | tr -d "\"")
declare -i COUNT=0

while [ "$CHANGE_STATUS" == "PENDING" ]
do
  COUNT=COUNT+1
  if [ "$COUNT" -ge 6 ]
  then
    echo "Update timed out, exiting..."
    exit 1
  fi
  sleep 10
  CHANGE_STATUS=$(aws route53 --output json get-change --id $CHANGE_ID | jq ".ChangeInfo.Status" | tr -d "\"")
done

echo "Record updated!"

*note: use “`set -ex“` option to debug the script

List of AWS regions and availability zones

List of  AWS Regions

This is complete list of  AWS regions available currently.

S.No Code Name
1 us-east-1 US East (N. Virginia)
2 us-west-2 US West (Oregon)
3 us-west-1 US West (N. California)
4 eu-west-1 EU (Ireland)
5 eu-central-1 EU (Frankfurt)
6 ap-southeast-1 Asia Pacific (Singapore)
7 ap-northeast-1 Asia Pacific (Tokyo)
8 ap-southeast-2 Asia Pacific (Sydney)
9 ap-northeast-2 Asia Pacific (Seoul)
10 sa-east-1 South America (São Paulo)
11 cn-north-1 China (Beijing)
12 ap-south-1 India (Mumbai)

AWS upcoming regions

 

S.No Code Name
1 N/A OHIO
2 N/A MONTREAL
3 N/A UK
4 N/A INDIA
5 N/A NINGXIA

List of  AWS regions and their availability zones

S.No AWS region code AWS region name Number Of Availability Zones Availability Zone Names
1 us-east-1 Virginia 4 us-east-1a
us-east-1b
us-east-1cus-east-1dus-east-1e

us-east-1f

2 us-west-2 Oregon 3 us-west-2a
us-west-2b
us-west-2c
3 us-west-1 N. California 3 us-west-1a
us-west-1b
4 eu-west-1 Ireland 3 eu-west-1a
eu-west-1b
eu-west-1c
5 eu-central-1 Frankfurt 2 eu-central-1a
eu-central-1b
6 ap-southeast-1 Singapore 2 ap-southeast-1a
ap-southeast-1b
7 ap-southeast-2 Sydney 3 ap-southeast-2a
ap-southeast-2b
ap-southeast-2c
8 ap-northeast-1 Tokyo 2 ap-northeast-1a
ap-northeast-1c
9 ap-northeast-2 Seoul N/A N/A
10 sa-east-1 Sao Paulo 3 sa-east-1a
sa-east-1b
sa-east-1c
11 cn-north-1 China (Beijing) N/A N/A
12 ap-south-1 India (Mumbai) 2 ap-south-1a
ap-south-1b

If you are familiar with AWS CLI you can always check regions and availability zones using following aws cli commands

Find regions using AWS CLI

Command:  aws ec2 describe-regions

aws ec2 describe-regions
{
    "Regions": [
        {
            "Endpoint": "ec2.eu-west-1.amazonaws.com", 
            "RegionName": "eu-west-1"
        }, 
        {
            "Endpoint": "ec2.ap-southeast-1.amazonaws.com", 
            "RegionName": "ap-southeast-1"
        }, 
        {
            "Endpoint": "ec2.ap-southeast-2.amazonaws.com", 
            "RegionName": "ap-southeast-2"
        }, 
        {
            "Endpoint": "ec2.eu-central-1.amazonaws.com", 
            "RegionName": "eu-central-1"
        }, 
        {
            "Endpoint": "ec2.ap-northeast-2.amazonaws.com", 
            "RegionName": "ap-northeast-2"
        }, 
        {
            "Endpoint": "ec2.ap-northeast-1.amazonaws.com", 
            "RegionName": "ap-northeast-1"
        }, 
        {
            "Endpoint": "ec2.us-east-1.amazonaws.com", 
            "RegionName": "us-east-1"
        }, 
        {
            "Endpoint": "ec2.sa-east-1.amazonaws.com", 
            "RegionName": "sa-east-1"
        }, 
        {
            "Endpoint": "ec2.us-west-1.amazonaws.com", 
            "RegionName": "us-west-1"
        }, 
        {
            "Endpoint": "ec2.us-west-2.amazonaws.com", 
            "RegionName": "us-west-2"
        }
    ]
}

 

Find AWS availability zones using AWS CLI

You can find the availability zones of particular region using following command

 aws ec2 describe-availability-zones --region us-west-1
{
    "AvailabilityZones": [
        {
            "State": "available", 
            "RegionName": "us-west-1", 
            "Messages": [], 
            "ZoneName": "us-west-1b"
        }, 
        {
            "State": "available", 
            "RegionName": "us-west-1", 
            "Messages": [], 
            "ZoneName": "us-west-1c"
        }
    ]
}

There are other two commands ec2-describe-regions and ec2-describe-availability-zone which are also helpful to retrieve regions and availability zones respectively. These are available in the package ec2-api-tools

You can check the availability zones of your current region in AWS console in the dashboard under service health, under availability zones

AWS Regions  google map

Find AWS Regions location here in google map (under development). You are invited to improve.

 

 

Note: AWS frequently updates availability zones and regions. Please consider also checking zones on aws console. 

References

[1]  http://docs.aws.amazon.com/general/latest/gr/rande.html
[2] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html
[3]https://aws.amazon.com/about-aws/global-infrastructure/