Speed up SSH with multiplexing

SSH multiplexing is the ability to carry multiple SSH sessions over a single TCP connection.

OpenSSH can reuse an existing TCP connection for multiple concurrent SSH sessions. This results into reduction of the overhead of creating new TCP connections.

Advantage of using SSH multiplexing is that it speeds up certain operations that rely on or occur over SSH. For example, let’s say that you’re using SSH to regularly execute a command on a remote host. Without multiplexing, every time that command is executed your SSH client must establish a new TCP connection and a new SSH session with the remote host. With multiplexing, you can configure SSH to establish a single TCP connection that is kept alive for a specific period of time, and SSH sessions are established over that connection.

You can see the difference below

without multiplexing, we see the normal connection time:

$ time ssh lintel-blog

Then we do the same thing again, but with a multiplexed connection to see a faster result:

$ time ssh lintel-blog

Configure Multiplexing

OpenSSH client supports multiplexing its outgoing connections, since version 3.9, using the ControlMaster, ControlPath and ControlPersist configuration directives which get defined in ssh_config. The client configuration file usually defaults to the location ~/.ssh/config.

ControlMaster determines whether ssh will listen for control connections and what to do about them. ControlPath sets the location for the control socket used by the multiplexed sessions. These can be either globally or locally in ssh_config or else specified at run time. Control sockets are removed automatically when the master connection has ended. ControlPersist can be used in conjunction with ControlMaster. If ControlPersist is set to ‘yes’, then it will leave the master connection open in the background to accept new connections until either killed explicitly or closed with -O or ends at a pre-defined timeout. If ControlPersist is set to a time, then it will leave the master connection open for the designated time or until the last multiplexed session is closed, whichever is longer.

Here is a sample excerpt from ssh_config applicable for starting a multiplexed session to server1.example.org via the shortcut server1.

 

Parallel command execution – Linux Cluster

The pdsh parallel shell tool allows you and lets you run a shell command across multiple nodes in a cluster.

This is a high performance, parallel pdsh shell remote shell utility for admins. Chaos Pdsh is a multithreaded remote shell client which executes commands on multiple remote hosts in parallel.  A parallel shell permits your clusters Linux Ubuntu RedHat to run the same similar command on many designated hosts or nodes within the hadoop cluster. In this case you do not have to really log in to each node individually.

High-performance and parallel remote shell utility with dshgroup module allows dsh on pdsh (or otherwise known as Dancer’s shell sudo) files from /etc/dsh/group directory. Now download Parallel Distributed Shell free of charge.

What is pdsh?

pdsh is a variant of the rsh(1) command. Unlike rsh(1), which runs commands on a single remote host, pdsh can run multiple remote commands in parallel. pdsh uses a “sliding window” (or fanout) of threads to conserve resources on the initiating host while allowing some connections to time out.

When pdsh receives SIGINT (ctrl-C), it lists the status of current threads. A second SIGINT within one second terminates the program. Pending threads may be canceled by issuing ctrl-Z within one second of ctrl-C. Pending threads are those that have not yet been initiated, or are still in the process of connecting to the remote host.

If a remote command is not specified on the command line, pdsh runs interactively, prompting for commands and executing them when terminated with a carriage return. In interactive mode, target nodes that time out on the first command are not contacted for subsequent commands, and commands prefixed with an exclamation point will be executed on the local system.

The core functionality of pdsh may be supplemented by dynamically loadable modules. The modules may provide a new connection protocol (replacing the standard rcmd(3) protocol used by rsh(1)), filtering options (e.g. removing hosts that are “down” from the target list), and/or host selection options (e.g., -a selects all hosts from a configuration file.). By default, pdsh must have at least one “rcmd” module loaded. See the RCMD MODULES section for more information.

Installing pdsh

Debian based:

apt install pdsh

RHEL based:

yum install pdsh

Running

The following command installs telegraf on all 4 nodes in cluster02

Running multiple commands

Pipe redirection

 

Example

 

When using ssh for remote execution, expect the stderr of ssh to be folded in with that of the remote command. When invoked by pdsh, it is not possible for ssh to prompt for passwords if RSA/DSA keys are configured properly, etc.. For ssh implementations that suppport a connect timeout option, pdsh attempts to use that option to enforce the timeout (e.g. -oConnectTimeout=T for OpenSSH), otherwise connect timeouts are not supported when using ssh. Finally, there is no reliable way for pdsh to ensure that remote commands are actually terminated when using a command timeout. Thus if -u is used with ssh commands may be left running on remote hosts even after timeout has killed local ssh processes.

Output from multiple processes per node may be interspersed when using qshell or mqshell rcmd modules.

The number of nodes that pdsh can simultaneously execute remote jobs on is limited by the maximum number of threads that can be created concurrently, as well as the availability of reserved ports in the rsh and qshell rcmd modules. On systems that implement Posix threads, the limit is typically defined by the constant PTHREADS_THREADS_MAX.

Manhole service in Twisted Application.

What is Manhole?

Manhole is an in-process service, that will accept UNIX domain socket connections and present the stack traces for all threads and an interactive prompt.

Using it we can access and modify objects or definition in the running application, like change or add the method in any class, change the definition of any method of class or module.

This allows us to make modifications in running an application without restarting the application, it makes work easy like debugging the application, you are able to check the values of the object while the program is running.

How to configure it?

Once you run above snippet, the service will start on TCP port 2222.

You need to use SSH command to get login into the service.

See below how it looks like.

Here In the first login, we change the value in DATA dictionary in running application, as we can see we get the new value in the second login.

Simple port scanner in python

a port scanner is an application designed to probe a server or host for open ports. Such an application may be used by administrators to verify the security policies of their networks and by attackers to identify network services running on a host and exploit vulnerabilities.

port-scanner.py

Example

port-scannin

Howto use ssh as VPN tunnel

SSH is typically used to log into a remote machine and execute commands, but it also supports tunneling, forwarding TCP ports and X11 connections.

What is SSH Tunneling?

A tunneling protocol may, for example, allow a foreign protocol to run over a network that does not support that particular protocol, such as running IPv6 over IPv4.

SSH tunneling is a method of transporting arbitrary networking data over an encrypted SSH connection. It can be used to add encryption to legacy applications. … It also provides a way to secure the data traffic of any given application using port forwarding, basically tunneling any TCP/IP port over SSH.

sshuttle

sshuttle is not exactly a VPN, and not exactly port forwarding. It’s kind of both, and kind of neither.

It’s like a VPN, since it can forward every port on an entire network, not just ports you specify. Conveniently, it lets you use the “real” IP addresses of each host rather than faking port numbers on localhost.

On the other hand, the way it works is more like ssh port forwarding than a VPN. Normally, a VPN forwards your data one packet at a time, and doesn’t care about individual connections; ie. it’s “stateless” with respect to the traffic. sshuttle is the opposite of stateless; it tracks every single connection.

Installation

 sudo pip install sshuttle

Example

$ sshuttle --dns -v -r <remote-host> 0/0

ssh-tunnel

* This will forward all connections including DNS requests…

Usage

Still Confused With SMTP, IMAP, POP Ports?

Configuring SMTP, IMAP and POP ports can be confusing. Users and sometimes even system administrators aren’t sure when to use port 25, 587, or 465.

This article will clarify all ports related to the mail server.

SMTP 25
SMTP-SSL/TLS 587,465
IMAP 143
IMAP-SSL/TLS 993
POP3 110
POP3-SSL/TLS 995

IMAP uses port 143, but SSL/TLS encrypted IMAP uses port 993.

POP uses port 110, but SSL/TLS encrypted POP uses port 995.

SMTP uses port 25, but SSL/TLS encrypted SMTP uses port 465.

587 vs. 465
These port assignments are specified by the Internet Assigned Numbers Authority (IANA):

Port 587: [SMTP] Message submission (SMTP-MSA), a service that accepts submission of email from email clients (MUAs). Described in RFC 6409.
Port 465: URL Rendezvous Directory for SSM (entirely unrelated to email)
Historically, port 465 was initially planned for the SMTPS encryption and authentication “wrapper” over SMTP, but it was quickly deprecated (within months, and over 15 years ago) in favor of STARTTLS over SMTP (RFC 3207). Despite that fact, there are probably many servers that support the deprecated protocol wrapper, primarily to support older clients that implemented SMTPS. Unless you need to support such older clients, SMTPS and its use on port 465 should remain nothing more than a historical footnote.

Howto reverse proxy in nginx

Proxying is typically used to distribute the load among several servers, seamlessly show content from different websites, or pass requests for processing to application servers over protocols other than HTTP.

When NGINX proxies a request, it sends the request to a specified proxied server, fetches the response, and sends it back to the client. It is possible to proxy requests to an HTTP server (another NGINX server or any other server) or a non-HTTP server (which can run an application developed with a specific framework, such as PHP or Python) using a specified protocol.

1. To pass a request to an HTTP proxied server, the proxy_pass directive is specified inside a location. For example:

 2. This address can be specified as a domain name or an IP address. The address may also include a port:

3. To pass a request to a non-HTTP proxied server, the appropriate **_pass directive should be used:

  • fastcgi_pass passes a request to a FastCGI server
  • uwsgi_pass passes a request to a uwsgi server
  • scgi_pass passes a request to an SCGI server
  • memcached_pass passes a request to a memcached server

4. Passing Request Headers

 

5. To disable buffering in a specific location, place the proxy_buffering directive in the location with the off parameter, as follows:

 

 

Openvas installation in CentOS 7

What is Openvas?

OpenVAS (Open Vulnerability Assessment System, originally known as GNessUs) is a software framework of several services and tools offering vulnerability scanning and vulnerability management.

All OpenVAS products are free software, and most components are licensed under the GNU General Public License (GPL). Plugins for OpenVAS are written in the Nessus Attack Scripting Language, NASL.

The primary reason to use this scan type is to perform comprehensive security testing of an IP address. It will initially perform a port scan of an IP address to find open services. Once listening services are discovered they are then tested for known vulnerabilities and misconfiguration using a large database (more than 53000 NVT checks). The results are then compiled into a report with detailed information regarding each vulnerability and notable issues discovered.

Once you receive the results of the tests, you will need to check each finding for relevance and possibly false positives. Any confirmed vulnerabilities should be re-mediated to ensure your systems are not at risk.

Vulnerability scans performed from externally hosted servers give you the same perspective as an attacker. This has the advantage of understanding exactly what is exposed on external-facing services.

Step 1: Disable SELinux

sed -i 's/=enforcing/=disabled/' /etc/selinux/config

and reboot the machine.

Step 2:  Install dependencies

yum -y install wget rsync curl net-tools

Step 3: Install OpenVAS repository

install the official repository so that OpenVAS works appropriately in the analysis of vulnerabilities.

wget -q -O - http://www.atomicorp.com/installers/atomic |sh

Step 4: Install OpenVAS

yum -y install openvas

Step 5: Run OpenVAS

Once OpenVAS is installed, we continue to start it by executing the following command:

openvas-setup

Once downloaded it will be necessary to configure the GSAD IP address, Greenbone Security Assistant, which is a web interface to manage system scans.

Step 6: Configure OpenVAS Connectivity

We go to our browser and enter the IP address of the CentOS 7 server where we have installed OpenVAS, and we will see that the following message is displayed:

Openvas dashboard

 

Automatic NVT Updates With Cron

35 1 * * * /usr/sbin/greenbone-nvt-sync > /dev/null
5 0 * * * /usr/sbin/greenbone-scapdata-sync > /dev/null
5 1 * * * /usr/sbin/greenbone-certdata-sync > /dev/null

 

How to use ipset command on linux to block bulk IPs

ipset is a companion application for the iptables Linux firewall. It allows you to setup rules to quickly and easily block a set of IP addresses, among other things.

Installation

Debian based system

# apt install ipset

Redhat based system

# yum install ipset

Blocking a list of network

Start by creating a new “set” of network addresses. This creates a new “hash” set of “net” network addresses named “myset”.

or

Add any IP address that you’d like to block to the set.

Finally, configure iptables to block any address in that set. This command will add a rule to the top of the “INPUT” chain to “-m” match the set named “myset” from ipset (–match-set) when it’s a “src” packet and “DROP”, or block, it.

Blocking a list of IP addresses

Start by creating a new “set” of ip addresses. This creates a new “hash” set of “ip” addresses named “myset-ip”.

or

Add any IP address that you’d like to block to the set.

Finally, configure iptables to block any address in that set.

Making ipset persistent

The ipset you have created is stored in memory and will be gone after reboot. To make the ipset persistent you have to do the followings:

First save the ipset to /etc/ipset.conf:

Then enable ipset.service, which works similarly to iptables.service for restoring iptables rules.

Other Commands

To view the sets:

or

To delete a set named “myset”:

or

To delete all sets:

How to configure IPsec/L2TP VPN Clients on Linux

After setting up your own VPN server, follow these steps to configure your devices. In case you are unable to connect, first, check to make sure the VPN credentials were entered correctly.

Commands must be run as root on your VPN client.

To set up the VPN client, first install the following packages:

Create VPN variables (replace with actual values):

Configure strongSwan:

Configure xl2tpd:

The VPN client setup is now complete. Follow the steps below to connect.

Note: You must repeat all steps below every time you try to connect to the VPN.

Create xl2tpd control file:

Restart services:

Start the IPsec connection:

Start the L2TP connection:

Run ifconfig and check the output. You should now see a new interface ppp0.

Check your existing default route:

Find this line in the output: default via X.X.X.X .... Write down this gateway IP for use in the two commands below.

Exclude your VPN server’s IP from the new default route (replace with actual value):

If your VPN client is a remote server, you must also exclude your Local PC’s public IP from the new default route, to prevent your SSH session from being disconnected (replace with actual value):

Add a new default route to start routing traffic via the VPN server:

The VPN connection is now complete. Verify that your traffic is being routed properly:

The above command should return Your VPN Server IP.

To stop routing traffic via the VPN server:

To disconnect: