Tag Archives: network

Manhole service in Twisted Application.

What is Manhole?

Manhole is an in-process service, that will accept UNIX domain socket connections and present the stack traces for all threads and an interactive prompt.

Using it we can access and modify objects or definition in the running application, like change or add the method in any class, change the definition of any method of class or module.

This allows us to make modifications in running an application without restarting the application, it makes work easy like debugging the application, you are able to check the values of the object while the program is running.

How to configure it?

from twisted.internet import reactor
from twisted.conch import manhole, manhole_ssh
from twisted.conch.ssh.keys import Key
from twisted.cred import portal, checkers

DATA = {"Service": "Manhole"}


def get_manhole_factory(namespace, **passwords):

    def get_manhole(arg):
        return manhole.ColoredManhole(namespace)
            
    realm = manhole_ssh.TerminalRealm()
    realm.chainedProtocolFactory.protocolFactory = get_manhole
    p = portal.Portal(realm)
    p.registerChecker(checkers.InMemoryUsernamePasswordDatabaseDontUse(**passwords))
    f = manhole_ssh.ConchFactory(p)
    f.publicKeys = {"ssh-rsa": Key.fromFile("keys/manhole.pub")}
    f.privateKeys = {"ssh-rsa": Key.fromFile("keys/manhole")}
    return f


reactor.listenTCP(2222, get_manhole_factory(globals(), admin='admin'))
reactor.run()

Once you run above snippet, the service will start on TCP port 2222.

You need to use SSH command to get login into the service.

See below how it looks like.

[lalit : ~]₹ ssh admin@localhost -p 2222
admin@localhost's password:
>>> dir() 
['DATA', '__builtins__', '__doc__', '__file__', '__name__', '__package__', 'checkers', 'get_manhole_factory', 'manhole', 'manhole_ssh', 'portal', 'reactor'] 
>>> DATA 
{'Service': 'Manhole'}
>>> DATA['Service'] = "Edited" 
>>> DATA 
{'Service': 'Edited'}
[lalit : ~]₹ ssh admin@localhost -p 2222
admin@localhost's password: 
>>> dir() 
['DATA', '__builtins__', '__doc__', '__file__', '__name__', '__package__', 'checkers', 'get_manhole_factory', 'manhole', 'manhole_ssh', 'portal', 'reactor'] 
>>> DATA 
{'Service': 'Edited'} 

Here In the first login, we change the value in DATA dictionary in running application, as we can see we get the new value in the second login.

How to use ipset command on linux to block bulk IPs

ipset is a companion application for the iptables Linux firewall. It allows you to setup rules to quickly and easily block a set of IP addresses, among other things.

Installation

Debian based system

“`# apt install ipset“`

Redhat based system

“`# yum install ipset“`

Blocking a list of network

Start by creating a new “set” of network addresses. This creates a new “hash” set of “net” network addresses named “myset”.

# ipset create myset hash:net

or

# ipset -N myset nethash

Add any IP address that you’d like to block to the set.

# ipset add myset 14.144.0.0/12
# ipset add myset 27.8.0.0/13
# ipset add myset 58.16.0.0/15
# ipset add myset 1.1.1.0/24

Finally, configure iptables to block any address in that set. This command will add a rule to the top of the “INPUT” chain to “-m” match the set named “myset” from ipset (–match-set) when it’s a “src” packet and “DROP”, or block, it.

# iptables -I INPUT -m set --match-set myset src -j DROP

Blocking a list of IP addresses

Start by creating a new “set” of ip addresses. This creates a new “hash” set of “ip” addresses named “myset-ip”.

# ipset create myset-ip hash:ip

or

# ipset -N myset-ip iphash

Add any IP address that you’d like to block to the set.

# ipset add myset-ip 1.1.1.1
# ipset add myset-ip 2.2.2.2

Finally, configure iptables to block any address in that set.

# iptables -I INPUT -m set --match-set myset-ip src -j DROP

Making ipset persistent

The ipset you have created is stored in memory and will be gone after reboot. To make the ipset persistent you have to do the followings:

First save the ipset to /etc/ipset.conf:

# ipset save > /etc/ipset.conf

Then enable ipset.service, which works similarly to iptables.service for restoring iptables rules.

Other Commands

To view the sets:

# ipset list

or

# ipset -L

To delete a set named “myset”:

# ipset destroy myset

or

# ipset -X myset

To delete all sets:

# ipset destroy

How to configure IPsec/L2TP VPN Clients on Linux

After setting up your own VPN server, follow these steps to configure your devices. In case you are unable to connect, first, check to make sure the VPN credentials were entered correctly.

Commands must be run as root on your VPN client.

To set up the VPN client, first install the following packages:

# For Ubuntu & Debian
apt-get update
apt-get -y install strongswan xl2tpd

# For RHEL/CentOS
yum -y install epel-release
yum --enablerepo=epel -y install strongswan xl2tpd

yum -y install strongswan xl2tpd

Create VPN variables (replace with actual values):

VPN_SERVER_IP=your_vpn_server_ip
VPN_IPSEC_PSK=your_ipsec_pre_shared_key
VPN_USER=your_vpn_username
VPN_PASSWORD=your_vpn_password

Configure strongSwan:

cat > /etc/ipsec.conf <<EOF
# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
  # strictcrlpolicy=yes
  # uniqueids = no

# Add connections here.

# Sample VPN connections

conn %default
  ikelifetime=60m
  keylife=20m
  rekeymargin=3m
  keyingtries=1
  keyexchange=ikev1
  authby=secret
  ike=aes128-sha1-modp2048!
  esp=aes128-sha1-modp2048!

conn myvpn
  keyexchange=ikev1
  left=%defaultroute
  auto=add
  authby=secret
  type=transport
  leftprotoport=17/1701
  rightprotoport=17/1701
  right=$VPN_SERVER_IP
EOF

cat > /etc/ipsec.secrets <<EOF
: PSK "$VPN_IPSEC_PSK"
EOF

chmod 600 /etc/ipsec.secrets

# For CentOS/RHEL & Fedora ONLY
mv /etc/strongswan/ipsec.conf /etc/strongswan/ipsec.conf.old 2>/dev/null
mv /etc/strongswan/ipsec.secrets /etc/strongswan/ipsec.secrets.old 2>/dev/null
ln -s /etc/ipsec.conf /etc/strongswan/ipsec.conf
ln -s /etc/ipsec.secrets /etc/strongswan/ipsec.secrets

Configure xl2tpd:

cat > /etc/xl2tpd/xl2tpd.conf <<EOF
[lac myvpn]
lns = $VPN_SERVER_IP
ppp debug = yes
pppoptfile = /etc/ppp/options.l2tpd.client
length bit = yes
EOF

cat > /etc/ppp/options.l2tpd.client <<EOF
ipcp-accept-local
ipcp-accept-remote
refuse-eap
require-chap
noccp
noauth
mtu 1280
mru 1280
noipdefault
defaultroute
usepeerdns
connect-delay 5000
name $VPN_USER
password $VPN_PASSWORD
EOF

chmod 600 /etc/ppp/options.l2tpd.client

The VPN client setup is now complete. Follow the steps below to connect.

Note: You must repeat all steps below every time you try to connect to the VPN.

Create xl2tpd control file:

mkdir -p /var/run/xl2tpd
touch /var/run/xl2tpd/l2tp-control

Restart services:

service strongswan restart
service xl2tpd restart

Start the IPsec connection:

# Ubuntu & Debian
ipsec up myvpn

# CentOS/RHEL & Fedora
strongswan up myvpn

Start the L2TP connection:

echo "c myvpn" > /var/run/xl2tpd/l2tp-control

Run ifconfig and check the output. You should now see a new interface ppp0.

Check your existing default route:

ip route

Find this line in the output: default via X.X.X.X .... Write down this gateway IP for use in the two commands below.

Exclude your VPN server’s IP from the new default route (replace with actual value):

route add YOUR_VPN_SERVER_IP gw X.X.X.X

If your VPN client is a remote server, you must also exclude your Local PC’s public IP from the new default route, to prevent your SSH session from being disconnected (replace with actual value):

route add YOUR_LOCAL_PC_PUBLIC_IP gw X.X.X.X

Add a new default route to start routing traffic via the VPN server:

route add default dev ppp0

The VPN connection is now complete. Verify that your traffic is being routed properly:

wget -qO- http://ipv4.icanhazip.com; echo

The above command should return Your VPN Server IP.

To stop routing traffic via the VPN server:

route del default dev ppp0

To disconnect:

# Ubuntu & Debian
echo "d myvpn" > /var/run/xl2tpd/l2tp-control
ipsec down myvpn

# CentOS/RHEL & Fedora
echo "d myvpn" > /var/run/xl2tpd/l2tp-control
strongswan down myvpn

How to specify the source address for all outbound connections

If you have multiple IPs assigned on your Linux pc then there is a chance that you want to use different IPs for some applications than default one. Updating IP routes every time isn’t a good idea and you may mess up.

get bindhack.c

“`

wget ‘https://gist.githubusercontent.com/akhilin/f6660a2f93f64545ff8fcc0d6b23e42a/raw/7bf3f066b74a4b9e3d3768a8affee26da6a3ada6/bindhack.c’ -P /tmp/

“`

compile it

“`

gcc -fPIC -static -shared -o /tmp/bindhack.so /tmp/bindhack.c -lc -ldl

“`

Copy it to library folder

“`

cp /tmp/bindhack.so /usr/lib/ && chmod +x /usr/lib/bindhack.so

“`

Optional (ignore if you have it already )

“`

echo ‘nameserver 8.8.8.8’ >> /etc/resolv.conf

“`

using bindhack

“`

BIND_ADDR=<source ip> LD_PRELOAD=/usr/lib/bindhack.so <command here>

“`

Example

 

you can add below function in your .bashrc to spin it at any time

 

bindhack() {
# Author: Akhil Jalagam
[ $# -lt 2 ] && echo "missing arguments: $0 [bind ip] [command with quotes]"
wget 'https://gist.githubusercontent.com/akhilin/f6660a2f93f64545ff8fcc0d6b23e42a/raw/7bf3f066b74a4b9e3d3768a8affee26da6a3ada6/bindhack.c' -P /tmp/
gcc -fPIC -static -shared -o /tmp/bindhack.so /tmp/bindhack.c -lc -ldl
[ -f /usr/lib/bindhack.so ] || `type -P cp` /tmp/bindhack.so /usr/lib/ && chmod +x /usr/lib/bindhack.so
[ -f /etc/resolv.conf ] && `type -P cp` /etc/resolv.conf /etc/resolv.conf.bak && echo 'nameserver 8.8.8.8' >> /etc/resolv.conf
[ $# -eq 2 ] && BIND_ADDR=$1 LD_PRELOAD=/usr/lib/bindhack.so $2
echo ''
`type -P cp` /etc/resolv.conf.bak /etc/resolv.conf
}

 

take a look at bindhack.c

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <string.h>
#include <dlfcn.h>

#include <arpa/inet.h>

/* 
   This is the address you want to force everything to use. It can be
   overriden at runtime by specifying the BIND_SRC environment 
   variable.
*/
#define SRC_ADDR	"192.168.0.1"

/* 
   LIBC_NAME should be the name of the library that contains the real
   bind() and connect() library calls. On Linux this is libc, but on
   other OS's such as Solaris this would be the socket library
*/
#define LIBC_NAME	"libc.so.6" 

#define YES	1
#define NO	0

int
bind(int sockfd, const struct sockaddr *my_addr, socklen_t addrlen)
{

	struct sockaddr_in src_addr;
	void	*libc;
	int	(*bind_ptr)(int, void *, int);
	int	ret;
	int	passthru;
	char 	*bind_src;

#ifdef DEBUG
	fprintf(stderr, "bind() override called for addr: %s\n", SRC_ADDR);
#endif

	libc = dlopen(LIBC_NAME, RTLD_LAZY);

	if (!libc)
	{
		fprintf(stderr, "Unable to open libc!\n");
		exit(-1);
	}

	*(void **) (&bind_ptr) = dlsym(libc, "bind");

	if (!bind_ptr)
	{
		fprintf(stderr, "Unable to locate bind function in lib\n");
		exit(-1);
	}
	
	passthru = YES;	/* By default, we just call regular bind() */

	if (my_addr==NULL)
	{
		/* If we get a NULL it's because we're being called
		   from the connect() hack */

		passthru = NO;

#ifdef DEBUG
		fprintf(stderr, "bind() Received NULL address.\n");
#endif

	}
	else
	{

		if (my_addr->sa_family == AF_INET)
		{
			struct sockaddr_in	myaddr_in;

			/* If this is an INET socket, then we spring to
			   action! */
			passthru = NO;

			memcpy(&myaddr_in, my_addr, addrlen);

			src_addr.sin_port = myaddr_in.sin_port;


		}
		else
		{
			passthru = YES;
		}

	}

	if (!passthru)
	{

#ifdef DEBUG
		fprintf(stderr, "Proceeding with bind hack\n");
#endif

		src_addr.sin_family = AF_INET;

		bind_src=getenv("BIND_SRC");

		/* If the environment variable BIND_SRC is set, then use
		   that as the source IP to bind instead of the hard-coded
		   SRC_ADDR one.
		*/
		if (bind_src)
		{
			ret = inet_pton(AF_INET, bind_src, &src_addr.sin_addr);
			if (ret<=0)
			{
				/* If the above failed, then try the
				   built in address. */

				inet_pton(AF_INET, SRC_ADDR, 
						&src_addr.sin_addr);
			}
		}
		else
		{
			inet_pton(AF_INET, SRC_ADDR, &src_addr.sin_addr);
		}


	/* Call real bind function */
		ret = (int)(*bind_ptr)(sockfd, 
					(void *)&src_addr, 
					sizeof(src_addr));
	}
	else
	{

#ifdef DEBUG
		fprintf(stderr, "Calling real bind unmolested\n");
#endif

	/* Call real bind function */
		ret = (int)(*bind_ptr)(sockfd, 
					(void *)my_addr, 
					addrlen);

	}
#ifdef DEBUG
	fprintf(stderr, "The real bind function returned: %d\n", ret);
#endif

	/* Clean up */
	dlclose(libc);

	return ret;

}

/* 
	Sometimes (alot of times) programs don't bother to call bind() 
	if they're just making an outgoing connection. To take care of
	these cases, we need to call bind when they call connect 
	instead. And of course, then call connect as well...
*/

int
connect(int  sockfd, const struct sockaddr *serv_addr, socklen_t addrlen)
{
	int	(*connect_ptr)(int, void *, int);
	void	*libc;
	int	ret;

#ifdef DEBUG
	fprintf(stderr, "connect() override called for addr: %s\n", SRC_ADDR);
#endif

	/* Before we call connect, let's call bind() and make sure we're
	   using our preferred source address.
	*/

	ret = bind(sockfd, NULL, 0); /* Our fake bind doesn't really need
					those params */

	libc = dlopen(LIBC_NAME, RTLD_LAZY);

	if (!libc)
	{
		fprintf(stderr, "Unable to open libc!\n");
		exit(-1);
	}

	*(void **) (&connect_ptr) = dlsym(libc, "connect");

	if (!connect_ptr)
	{
		fprintf(stderr, "Unable to locate connect function in lib\n");
		exit(-1);
	}


	/* Call real connect function */
	ret = (int)(*connect_ptr)(sockfd, (void *)serv_addr, addrlen);

	/* Clean up */
	dlclose(libc);

	return ret;	

}

 

 

Network namespaces – part 2

How to run OpenVPN tunnel inside a network namespace

Linux network namespaces can be used to control which processes should be tunneled by OpenVPN.

First create an –up and –down script for OpenVPN. This script will create the VPN tunnel interface inside a network namespace called vpn, instead of the default namespace.

cat > netns-up << EOF
#!/bin/sh
case $script_type in
        up)
                ip netns add vpn
                ip netns exec vpn ip link set dev lo up
                mkdir -p /etc/netns/vpn
                echo "nameserver 8.8.8.8" > /etc/netns/vpn/resolv.conf
                ip link set dev "$1" up netns vpn mtu "$2"
                ip netns exec vpn ip addr add dev "$1" \
                        "$4/${ifconfig_netmask:-30}" \
                        ${ifconfig_broadcast:+broadcast "$ifconfig_broadcast"}
                test -n "$ifconfig_ipv6_local" && \
          ip netns exec vpn ip addr add dev "$1" \
                        "$ifconfig_ipv6_local"/112
                ;;
        route-up)
                ip netns exec vpn ip route add default via "$route_vpn_gateway"
                test -n "$ifconfig_ipv6_remote" && \
          ip netns exec vpn ip route add default via \
                        "$ifconfig_ipv6_remote"
                ;;
        down)
                ip netns delete vpn
                ;;
esac
EOF

Then start OpenVPN and tell it to use our –up script instead of executing ifconfig and route.

openvpn --ifconfig-noexec --route-noexec --up netns-up --route-up netns-up --down netns-up

Now you can start programs to be tunneled like this:

ip netns exec vpn command

Or start a separate shell

ip netns exec vpn command

 

Network namespaces – part 1

Linux namespaces are a relatively new kernel feature which is essential for implementation of containers. A namespace wraps a global system resource into an abstraction which will be bound only to processes within the namespace, providing resource isolation. In this article I discuss network namespace and show a practical example.

What is namespace?

A namespace is a way of scoping a particular set of identifiers. Using a namespace, you can use the same identifier multiple times in different namespaces. You can also restrict an identifier set visible to particular processes.

For example, Linux provides namespaces for networking and processes, among other things. If a process is running within a process namespace, it can only see and communicate with other processes in the same namespace. So, if a shell in a particular process namespace ran ps waux, it would only show the other processes in the same namespace.

Linux network namespaces

In a network namespace, the scoped ‘identifiers’ are network devices; so a given network device, such as eth0, exists in a particular namespace. Linux starts up with a default network namespace, so if your operating system does not do anything special, that is where all the network devices will be located. But it is also possible to create further non-default namespaces, and create new devices in those namespaces, or to move an existing device from one namespace to another.

Each network namespace also has its own routing table, and in fact this is the main reason for namespaces to exist. A routing table is keyed by destination IP address, so network namespaces are what you need if you want the same destination IP address to mean different things at different times – which is something that OpenStack Networking requires for its feature of providing overlapping IP addresses in different virtual networks.

Each network namespace also has its own set of iptables (for both IPv4 and IPv6). So, you can apply different security to flows with the same IP addressing in different namespaces, as well as different routing.

Any given Linux process runs in a particular network namespace. By default this is inherited from its parent process, but a process with the right capabilities can switch itself into a different namespace; in practice this is mostly done using the ip netns exec NETNS COMMAND… invocation, which starts COMMAND running in the namespace named NETNS. Suppose such a process sends out a message to IP address A.B.C.D, the effect of the namespace is that A.B.C.D will be looked up in that namespace’s routing table, and that will determine the network device that the message is transmitted through.

Lets play with ip namespaces

By convention a named network namespace is an object at /var/run/netns/NAME that can be opened. The file descriptor resulting from opening /var/run/netns/NAME refers to the specified network namespace.

create a namespace

power up loopback device

open up a namespace shell

now we can use this shell like user shell where it uses ns1 namespace only

 

In part-2  , I will explain how to connect to internet from ns1 namespace and adding custom routes.

5 Ways to Speed Up SSH Connections in Linux

SSH is the most popular and secure method for managing Linux servers remotely. One of the challenges with remote server management is connection speeds, especially when it comes to session creation between the remote and local machines.

There are several bottlenecks to this process, one scenario is when you are connecting to a remote server for the first time; it normally takes a few seconds to establish a session. However, when you try to start multiple connections in succession, this causes an overhead (combination of excess or indirect computation time, memory, bandwidth, or other related resources to carry out the operation).

In this article, we will share four useful tips on how to speed up remote SSH connections in Linux.

1.Use Compression option in SSH

From the ssh man page (type man ssh to see the whole thing):

 -C      Requests compression of all data (including stdin, stdout,
         stderr, and data for forwarded X11 and TCP connections).  The
         compression algorithm is the same used by gzip(1), and the
         “level” can be controlled by the CompressionLevel option for pro-
         tocol version 1.  Compression is desirable on modem lines and
         other slow connections, but will only slow down things on fast
         networks.  The default value can be set on a host-by-host basis
         in the configuration files; see the Compression option.

 

ssh -C username@example.com

2.Force SSH Connection Over IPV4

OpenSSH supports both IPv4/IP6, but at times IPv6 connections tend to be slower. So you can consider forcing ssh connections over IPv4 only, using the syntax below:

# ssh -4 username@example.com

Alternatively, use the AddressFamily (specifies the address family to use when connecting) directive in your ssh configuration file  (global configuration) or ~/.ssh/config (user specific file).

The accepted values are “any”, “inet” for IPv4 only, or “inet6”.

AddressFamily inet

3. Reuse SSH Connection

An ssh client program is used to establish connections to an sshd daemon accepting remote connections. You can reuse an already-established connection when creating a new ssh session and this can significantly speed up subsequent sessions.

You can enable this in your ~/.ssh/config file.

ControlMaster auto
ControlPath /home/akhil/.ssh/sockets/ssh_mux_%x_%p_%r
ControlPersist yes

openssh doesn’t support %x(ip address in control paths),  use my repo instead

https://github.com/akhilin/openssh-portable.git

or use %h to use hostname instead of ip address

using ip address is recommended so that even if you connect using different hostnames it uses same socket ( very useful when using ansible , pdsh )

4. Use Specific SSH Authentication Method

Another way of speeding up ssh connections is to use a given authentication method for all ssh connections, and here we recommend configuring ssh passwordless login using ssh keygen in 5 easy steps.

Once that is done, use the PreferredAuthentications directive, within ssh_config files (global or user specific) above. This directive defines the order in which the client should try authentication methods (you can specify a command separated list to use more than one method).

PreferredAuthentications=publickey

If you prefer password authentication which is deemed unsecure, use this.

ssh -o "PreferredAuthentications=password" username@example.com

5.Disable DNS Lookup On Remote Machine

By default, sshd daemon looks up the remote host name, and also checks that the resolved host name for the remote IP address maps back to the very same IP address. This can result into delays in connection establishment or session creation.

The UseDNS directive controls the above functionality; to disable it, search and uncomment it in the /etc/ssh/sshd_config file. If it’s not set, add it with the value no.

UseDNS=no