How does email work ?

Most of people in this generation would spend their time daily to send and receive emails. E-mail plays a vital role in our daily activities. As technology evolved email became the one of major communication tool. It may not cost you to send or receive a mail but there is an intricate process involved behind. You just need to press  send button to send your message, your message goes through the complex machanism to get received by recipient.

The invention of email was started in 1961. There were more than couple of people to list as the inventors of email.

The evolution of email started from messaging users on the same computer, then message transition between computer then message among multi user and multi computer, finally email. Email has became revolutionary tool for communication. Now a days it is part of our daily life. We will see how email really works.

Terminology

There are different components involved in the email system. There are few abbreviated terms. Having idea about terminology and abbreviations  will help you to better understand the system.

MSA Mail Submission Agent
MTA Mail Transfer Agent
MDA Mail Delivery Agent
MX Mail Exchange
DNS Domain Name System

MUA is the software that user will use either to send or retrieve  the mail(message) from the server.

MSA is the piece of software which is installed on mail server. Where it is responsible to transfer the message to destination or mail server called MTA (Mail Transfer Agent)

MTA  is the Mail Transfer Agent. It is the piece software on the server which is responsible to route the mail to destination mail server. So we called it as mail router, mail server etc.                                                                        Here are few popular  MTA(Mail Server) softwares : postfixqmailCourier Mail and sendmail.

postfix is the one which is widely used, and it comes with many linux distributions.

You can find the exhaustive mail server softwares list here

Protocols involved

There are different protocol involved in email system. All of them are required to get email delivered to the recipient. They are building blocks of email system. Those protocols are,

SMTP Simple Mail Transport Protocol
IMAP Internet Message Access Protocol
POP Post Office Protocol
DNS Domain Name System (Protocol)

How does email work?

This sections will get you the idea about how these protocols work together to deliver the mail to recipient. Here is the abstract overview of email system.

How does email work abstract
How does email work abstract

The above figure will give you simple abstract overview of email system.  As described in the figure. SMTP is the protocol which is used to push or send an email to server by sender. IMAP and POP are the protocols which are used to check or retrieve message from server by recipient.  The recipient MUA is configured to use either IMAP or POP or BOTH (IMAP & POP). The protocol IMPA is bidirectional where POP is unidirectional. We will see more about POP and IMAP in later section.

        As per the above figure(abstract overview) you can sense that, the sender email client(MUA) will send message using SMPT protocol to mail server (MTA). The mail server will check for the destination if it finds the one it will connect to destination mail server, that is other MTA/MDA and will pass the message(mail) using SMTP protocol. The recipient server will store the received mail locally. Later if recipient will check the mail by using his mail client by connecting to his mail server.

Now let’s see flow of email in detail with following figure

Here Alice is sending  email to Bob(bob@domain.com) by using her email client. Where she will push her message to server using SMTP to send Bob. The mail server will determine the destination by getting the MX record for the destination server. The Alice mail server will check for the domain after in the to(recipient)  mail address to get the MX record. Once Alice(sender) mail server received the result from DNS for the DNS query to get MX record, this server will connect to destination mail server and will delivery the mail (message) using SMTP protocol. The destination mail server will store the message locally. Bob will check the newly arrived mails by using his mail client.

I would like to illustrate this process even more by using following figure.

In the above figure there are new components introduced, those are MSA, MDA and extra MX servers.

       MSA is the Mail Submission agent where it is piece of software it will receive the message from MUA. It uses the same protocol SMTP and port 25. Practically most MTAs perform the function of MSA so you may assume MTA as MSA.

       The MDA means mail delivery agent or message delivery agent  is a computer software component that is responsible for the delivery of e-mail messages to a local recipient’s mailbox. You may not need to get confused, most of MTAs perform the MDA functionality as well though there are softwares which are only designed to work as MDA.

Here, you can see that practically there will be more than 1 mail server(MX SERVER)s. All other MX servers are for backup purpose. When sender query DNS for MX record it may get more that one MX server with priorities. Here is the sample output of DNS query for gmail MX records. Gmail is having 5 MX servers with different priorities.

The above result is the out put of command line utility dig. Where we use this utility to query the DNS. In the output you would see something like below in the right side of  ANSWER SECTION.

Here you would see some numbers in front of domains. Those numbers will decide the priority of MX record. The domain which is having very lowest number associated will have highest priority. Here gmail-smtp-in.l.google.com MX record having highest priority cause it having lowest number associated, that is 5. So the sender MTA will try to connect with most prioritized MX first. If it is down or so, it will try to connect with next MX (Mail Exchange server). As a result mail will be delivered with no down time.

Once the mail is received by the destination mail server, it will store the message in the mail store. There are two types of mail stores used by various mail servers(softwares). Those are,

  • Mail DIR
  • Mail Box

Locally stored mail will be accessed(fetched) by the recipient client using either POP or IMAP.

Protocols POP and IMAP

POP and IMAP are application layer protocols, as described above POP and IMAP are the protocols which are used to fetch or access the mails by recipient email client (MUA). Both protocols are different, they server for different purpose.

The protocol POP means Post Office Protocol and POP3 is it’s  version 3.  As the name describes, this protocol is used to download the message from the server by the client. Once message is downloaded form the server, it will be removed unless you set the flag leave a copy on server just how post card is delivered to destination. If you are only the one who will access mailbox from one location the POP3 suites well. This will also save some memory on server. So, if use POP3 you can’t access the mail using different clients.

Unlike POP3 the IMAP won’t download the message by deleting it on server. It will just access the message like browser does webpages. So it is handy if you use multiple clients from different locations.

 

References:

[1] https://tools.ietf.org/html/rfc3501
[2] https://www.ietf.org/rfc/rfc1939.txt
[3] https://tools.ietf.org/html/rfc5321

How to assign IP to network interface command line in linux

This article objective is provide you the way to assign the IP address to the network interface from the command line.

You can see the available network interfaces on your machine by using the simple command ifconfig

You can find the interfaces and their names and you can identify the name which is  preceded by colon(:) in the left side. Something like eth0, lo and wlan0 etc.  The network interface lo interface is the special interface where we called it as localhost and special IP assigned to it 127.0.0.1 also called loopback IP.

Do you know? you can assign multiple IPs to single network interface. This is pretty useful when you need multiple IP addresses but you have only one network card.

We created new interface alias with IP assigned. We can do it simply by giving colon(:) and alias number.

Assign IP using ifconfig

This command will assign the specified IP to the give network interface. It is not a persistent change. The would change after reboot.

 

Get IP using dhclient

The program dhclient will get you the IP to the given interface using the Dynamic Host Configuration Protocol (DHCP). The IP which will be assigned to the given network interface which will be provided by gateway or router. If you won’t have any preference of having specific IP assigned to the network card, this can be used

 

Assign static IP

The IP we assigned above is not a persistent or static IP. Means you will lose that IP and will get an other different IP assigned after reboot. To make this IP permanent we have to edit the configuration files.

Ubuntu /etc/network/interfaces

 

CentOS /etc/sysconfig/network-scripts/ifcfg-eth0

Unlike the distribution Debian, CentOS maintains the configuration in a separate file for each interface. The  file path would be something like  /etc/sysconfig/network-scripts/ifcfg-<interface_name>.

 

How to modify permissions(booleans) of SELinux for deamons(programs)

SELinux offers more enhanced security for linux. It is always recommended not to disable SELinux for servers which are more delicate, instead you can control the permissions for the deamons, programms or users using SELinux.

SELinux maintains the status of permissions for all deamons with attributes called booleans.

Get SELinux booleans

The above command will give you lot of variables with status either on or off. If you want to fetch for particular process or context use grep 

To get all booleans regarding httpd(apache web server)

 

Set SELinux Booleans

To set selinux booleans we use the command setsebool

Here is how you can change

For suppose if you want to allow httpd to allow sending mail

If you want to enable ftp server on httpd,

To enable apache to connect with external database

Like wise you can change the required booleans status. To query the modified status ues getsebool with grep.

 

How to Interactively discard selected changes in git

GIT is offering us the way to discard selected changes interactively. Here is how you will do it,

To checkout or discard selected changes from all modified files

git checkout -p  

If you want to checkout or discard selected changes from specific file, give that file as an argument

git checkout -p  file_name

As per this command git will break down the file into chunks called hunks. After you execute this command, you will be prompted with following question with several options to choose.

Discard this hunk from worktree [y,n,q,a,d,/,e,?]?

Here is the description of each option,

y – discard this hunk from worktree
n – do not discard this hunk from worktree
q – quit; do not discard this hunk nor any of the remaining ones
a – discard this hunk and all later hunks in the file
d – do not discard this hunk nor any of the later hunks in the file
g – select a hunk to go to
/ – search for a hunk matching the given regex
j – leave this hunk undecided, see next undecided hunk
J – leave this hunk undecided, see next hunk
k – leave this hunk undecided, see previous undecided hunk
K – leave this hunk undecided, see previous hunk
s – split the current hunk into smaller hunks
e – manually edit the current hunk
? – print help

You can also unstage the selected changes from all staged files or from specific file.

To unstage selected changes from all staged files use following command

git reset HEAD -p

To unstage changes from specific file give that file as an argument.

git reset HEAD -p  file_name

Here also you will be prompted to choose any one of the option to unstage hunk.

 

GIT Tips

List Contributed Users

We can list the user who contributed in git using command git log.  Here is the way to get,

And you can check users who contributed using following command

Enable Debugging

You can see curl debug message while push, pull and clone if you set the environment variable GIT_CURL_VERBOSE to 1

$ export GIT_CURL_VERBOSE=1

Set this variable before you push or pull or clone. You would be able to see debug message

List History Of Operations

The following command can be useful to list all history of operations performed using git

 

Stage all modified files in git

Stage all modified files using single command.

How to delete records by disabling foreign key checks or constraints in mysql

If you have a foreign key constraints on your table, you can’t delete records from parent table, but you can delete from child table. The main reason we use the foreign key constraints is to maintain data integrity.

Some times you many need to disable this kind of enforcing against some operations like delete. To do so, you have to set mysql server system variable foreign_key_checks

Now, you can delete or update with out any restriction because of foreign key constraint. But you will mess up the data integrity.

You can reset it back to enforce for integrity

There are global and local system variables. If you set the global variables it applies to all clients which are connected. If you change the local variable it applies to that current client session only. Here is how you can query local and global variable.

To sent global variable you just need to add global , where to set local you need to local. By default local or session variable will be changed

To set global variable

NOTE: Make sure you to revert the variable back to original state to maintain data integrity in your database.

 

How to enable or disable SELinux and check status on centOS

The SELinux stands for Security-Enhanced Linux where it is a linux kernel security module. It is enabled by default on most of the linux distribution that we use for servers like centOS. It provides  enhanced security measurements. It gives you fine control over all programs and daemons  on their activities like communicating with out side programs  or controlling whether to establish a outside connections for a particular program.

It is always recommended to have SELinux enabled on a server to avoid common security glitches.

To query the current status of SELinux  use the following commands

The above command will report the current status  of SELinux. Whether SELinux is enforcing, permissive, or disabled. If it is already disabled.

Disabling SELinux

Open the file  /etc/selinux/config  and change the option SELINUX to disabled

if you open file you would see something like

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing – SELinux security policy is enforced.
# permissive – SELinux prints warnings instead of enforcing.
# disabled – No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted – Targeted processes are protected,
# mls – Multi Level Security protection.
SELINUXTYPE=targeted

If SELinux is enabled you would see enforcing replace it with disabled.

You should restart the machine to take effect If you change SELINUX status from Enabled to Disabled or vice versa.

Enabling SELInux

To enable SELinux follow the below instructions,

  1. Open the file /etc/selinux/config  
  2. Change option SELINUX from disabled to enforcing
  3. Restart the machine

Change mode

To change the mode of SELinux which is running

Check Status

SELinux is the linux  kernel module for enhanced security. SELinux stands for Security-Enhanced Linux. If SELinux is installed on your machine or server you can check the current status  by using following commands

The above command will give you one of the following as an output

You use the below command which will give simple overview

If enabled you will output something like

 

You can also check the configuration which is located at /etc/selinux/config.

In above config file  the option SELINUX  describes the status of SELinux. But it’s not precise to determine the status from the configuration file, it’s better to determine the status by using commands mentioned above.

 

 

Note: You need administrator privileges to either enable or disable SELinux

SIP Request Methods Response Codes

SIP Request Methods Response Codes.

SIP Request Methods

There are several different Request methods to server different purposes. SIP borrowed headers and body format from the protocol HTTP. Like HTTP SIP also has different methods. following table will describe those request methods

 

SIP Request Methods
S.No Method Description
1 REGISTER Registers to receive inbound calls on registrar or SIP server
2 INVITE Established a new session
3 ACK Confirms that message/request has been received
4 BYE Ends Session
5 CANCEL Cancels establishing session
6 OPTIONS Queries capabilities of server
7 PARK Provisional Acknowledgement
8 SUBSCRIBE Subsribes for an Notifications from the Notifier
9 NOTIFY Notify the subscriber a new Event
10 PUBLISH Publish and event to the Server
11 INFO Sends mid-session information
12 REFERER Asks the recipient to issue call transfer
13 MESSAGE Transport Insant Message
14 UPDATE Modifies the state of a session without changing the state of the dialog.

….

SIP Responses

Like I mentioned earlier, SIP protocol borrowed few things from protocol HTTP. SIP borrowed response code from HTTP. Most of response codes are similar to HTTP and  SIP extends them codes to 6xx.

SIP Response Codes
S.No Code Description
1 1xx Informational responses
2 2xx Success Responses
3 3xx Redirect Responses
4 4xx Request Failure/Client Error
5 5xx Server Error
6 6xx Global Failures

How to send email command line in linux

We all know that email plays a vital role in our communication system. We will see how to send mail from command line in this article.

Assuming that you have postfix instaled on your machine. You can send the mail from command line by connecting to your local mail server. You will get the utility sendmail with postfix.

send email using sendmail

Send email using telnet

We all now about telnet command. It is very helpful to debug and understand the protocols. Here is the simple mail sent using telnet

In the above example we connected to the SMTP server as a client which is running on localhost. We gave commands to the server to send a mail. Please note that  the data ends with   <CR><LF>.<CR><LF> . to indicated it is the end of data. As you can see  . (dot) we mentioned  at the end.

Once we connected, we pass the set commands to server to write and send a mail. Here are the few SMTP commands

SMTP Commands

  •  HELO
  • MAIL
  • RCPT
  • DATA
  • RSET
  • VRFY
  • EXPN
  • HELP
  • NOOP
  • QUIT

We used MAIL, RCPT and QUIT commands in the above example. To understand these commands go through rfc . You may not need to understand low level protocol concepts of SMTP unless if you are really interested or you are working with SMTP.

References:

[1] http://tools.ietf.org/html/rfc5321