Manhole service in Twisted Application.

What is Manhole?

Manhole is an in-process service, that will accept UNIX domain socket connections and present the stack traces for all threads and an interactive prompt.

Using it we can access and modify objects or definition in the running application, like change or add the method in any class, change the definition of any method of class or module.

This allows us to make modifications in running an application without restarting the application, it makes work easy like debugging the application, you are able to check the values of the object while the program is running.

How to configure it?

Once you run above snippet, the service will start on TCP port 2222.

You need to use SSH command to get login into the service.

See below how it looks like.

Here In the first login, we change the value in DATA dictionary in running application, as we can see we get the new value in the second login.

Simple port scanner in python

a port scanner is an application designed to probe a server or host for open ports. Such an application may be used by administrators to verify the security policies of their networks and by attackers to identify network services running on a host and exploit vulnerabilities.

port-scanner.py

Example

port-scannin

Howto use ssh as VPN tunnel

SSH is typically used to log into a remote machine and execute commands, but it also supports tunneling, forwarding TCP ports and X11 connections.

What is SSH Tunneling?

A tunneling protocol may, for example, allow a foreign protocol to run over a network that does not support that particular protocol, such as running IPv6 over IPv4.

SSH tunneling is a method of transporting arbitrary networking data over an encrypted SSH connection. It can be used to add encryption to legacy applications. … It also provides a way to secure the data traffic of any given application using port forwarding, basically tunneling any TCP/IP port over SSH.

sshuttle

sshuttle is not exactly a VPN, and not exactly port forwarding. It’s kind of both, and kind of neither.

It’s like a VPN, since it can forward every port on an entire network, not just ports you specify. Conveniently, it lets you use the “real” IP addresses of each host rather than faking port numbers on localhost.

On the other hand, the way it works is more like ssh port forwarding than a VPN. Normally, a VPN forwards your data one packet at a time, and doesn’t care about individual connections; ie. it’s “stateless” with respect to the traffic. sshuttle is the opposite of stateless; it tracks every single connection.

Installation

 sudo pip install sshuttle

Example

$ sshuttle --dns -v -r <remote-host> 0/0

ssh-tunnel

* This will forward all connections including DNS requests…

Usage

Still Confused With SMTP, IMAP, POP Ports?

Configuring SMTP, IMAP and POP ports can be confusing. Users and sometimes even system administrators aren’t sure when to use port 25, 587, or 465.

This article will clarify all ports related to the mail server.

SMTP 25
SMTP-SSL/TLS 587,465
IMAP 143
IMAP-SSL/TLS 993
POP3 110
POP3-SSL/TLS 995

IMAP uses port 143, but SSL/TLS encrypted IMAP uses port 993.

POP uses port 110, but SSL/TLS encrypted POP uses port 995.

SMTP uses port 25, but SSL/TLS encrypted SMTP uses port 465.

587 vs. 465
These port assignments are specified by the Internet Assigned Numbers Authority (IANA):

Port 587: [SMTP] Message submission (SMTP-MSA), a service that accepts submission of email from email clients (MUAs). Described in RFC 6409.
Port 465: URL Rendezvous Directory for SSM (entirely unrelated to email)
Historically, port 465 was initially planned for the SMTPS encryption and authentication “wrapper” over SMTP, but it was quickly deprecated (within months, and over 15 years ago) in favor of STARTTLS over SMTP (RFC 3207). Despite that fact, there are probably many servers that support the deprecated protocol wrapper, primarily to support older clients that implemented SMTPS. Unless you need to support such older clients, SMTPS and its use on port 465 should remain nothing more than a historical footnote.

Howto reverse proxy in nginx

Proxying is typically used to distribute the load among several servers, seamlessly show content from different websites, or pass requests for processing to application servers over protocols other than HTTP.

When NGINX proxies a request, it sends the request to a specified proxied server, fetches the response, and sends it back to the client. It is possible to proxy requests to an HTTP server (another NGINX server or any other server) or a non-HTTP server (which can run an application developed with a specific framework, such as PHP or Python) using a specified protocol.

1. To pass a request to an HTTP proxied server, the proxy_pass directive is specified inside a location. For example:

 2. This address can be specified as a domain name or an IP address. The address may also include a port:

3. To pass a request to a non-HTTP proxied server, the appropriate **_pass directive should be used:

  • fastcgi_pass passes a request to a FastCGI server
  • uwsgi_pass passes a request to a uwsgi server
  • scgi_pass passes a request to an SCGI server
  • memcached_pass passes a request to a memcached server

4. Passing Request Headers

 

5. To disable buffering in a specific location, place the proxy_buffering directive in the location with the off parameter, as follows:

 

 

Openvas installation in CentOS 7

What is Openvas?

OpenVAS (Open Vulnerability Assessment System, originally known as GNessUs) is a software framework of several services and tools offering vulnerability scanning and vulnerability management.

All OpenVAS products are free software, and most components are licensed under the GNU General Public License (GPL). Plugins for OpenVAS are written in the Nessus Attack Scripting Language, NASL.

The primary reason to use this scan type is to perform comprehensive security testing of an IP address. It will initially perform a port scan of an IP address to find open services. Once listening services are discovered they are then tested for known vulnerabilities and misconfiguration using a large database (more than 53000 NVT checks). The results are then compiled into a report with detailed information regarding each vulnerability and notable issues discovered.

Once you receive the results of the tests, you will need to check each finding for relevance and possibly false positives. Any confirmed vulnerabilities should be re-mediated to ensure your systems are not at risk.

Vulnerability scans performed from externally hosted servers give you the same perspective as an attacker. This has the advantage of understanding exactly what is exposed on external-facing services.

Step 1: Disable SELinux

sed -i 's/=enforcing/=disabled/' /etc/selinux/config

and reboot the machine.

Step 2:  Install dependencies

yum -y install wget rsync curl net-tools

Step 3: Install OpenVAS repository

install the official repository so that OpenVAS works appropriately in the analysis of vulnerabilities.

wget -q -O - http://www.atomicorp.com/installers/atomic |sh

Step 4: Install OpenVAS

yum -y install openvas

Step 5: Run OpenVAS

Once OpenVAS is installed, we continue to start it by executing the following command:

openvas-setup

Once downloaded it will be necessary to configure the GSAD IP address, Greenbone Security Assistant, which is a web interface to manage system scans.

Step 6: Configure OpenVAS Connectivity

We go to our browser and enter the IP address of the CentOS 7 server where we have installed OpenVAS, and we will see that the following message is displayed:

Openvas dashboard

 

Automatic NVT Updates With Cron

35 1 * * * /usr/sbin/greenbone-nvt-sync > /dev/null
5 0 * * * /usr/sbin/greenbone-scapdata-sync > /dev/null
5 1 * * * /usr/sbin/greenbone-certdata-sync > /dev/null

 

How to use ipset command on linux to block bulk IPs

ipset is a companion application for the iptables Linux firewall. It allows you to setup rules to quickly and easily block a set of IP addresses, among other things.

Installation

Debian based system

# apt install ipset

Redhat based system

# yum install ipset

Blocking a list of network

Start by creating a new “set” of network addresses. This creates a new “hash” set of “net” network addresses named “myset”.

or

Add any IP address that you’d like to block to the set.

Finally, configure iptables to block any address in that set. This command will add a rule to the top of the “INPUT” chain to “-m” match the set named “myset” from ipset (–match-set) when it’s a “src” packet and “DROP”, or block, it.

Blocking a list of IP addresses

Start by creating a new “set” of ip addresses. This creates a new “hash” set of “ip” addresses named “myset-ip”.

or

Add any IP address that you’d like to block to the set.

Finally, configure iptables to block any address in that set.

Making ipset persistent

The ipset you have created is stored in memory and will be gone after reboot. To make the ipset persistent you have to do the followings:

First save the ipset to /etc/ipset.conf:

Then enable ipset.service, which works similarly to iptables.service for restoring iptables rules.

Other Commands

To view the sets:

or

To delete a set named “myset”:

or

To delete all sets:

How to configure IPsec/L2TP VPN Clients on Linux

After setting up your own VPN server, follow these steps to configure your devices. In case you are unable to connect, first, check to make sure the VPN credentials were entered correctly.

Commands must be run as root on your VPN client.

To set up the VPN client, first install the following packages:

Create VPN variables (replace with actual values):

Configure strongSwan:

Configure xl2tpd:

The VPN client setup is now complete. Follow the steps below to connect.

Note: You must repeat all steps below every time you try to connect to the VPN.

Create xl2tpd control file:

Restart services:

Start the IPsec connection:

Start the L2TP connection:

Run ifconfig and check the output. You should now see a new interface ppp0.

Check your existing default route:

Find this line in the output: default via X.X.X.X .... Write down this gateway IP for use in the two commands below.

Exclude your VPN server’s IP from the new default route (replace with actual value):

If your VPN client is a remote server, you must also exclude your Local PC’s public IP from the new default route, to prevent your SSH session from being disconnected (replace with actual value):

Add a new default route to start routing traffic via the VPN server:

The VPN connection is now complete. Verify that your traffic is being routed properly:

The above command should return Your VPN Server IP.

To stop routing traffic via the VPN server:

To disconnect:

How to specify the source address for all outbound connections

If you have multiple IPs assigned on your Linux pc then there is a chance that you want to use different IPs for some applications than default one. Updating IP routes every time isn’t a good idea and you may mess up.

get bindhack.c

wget 'https://gist.githubusercontent.com/akhilin/f6660a2f93f64545ff8fcc0d6b23e42a/raw/7bf3f066b74a4b9e3d3768a8affee26da6a3ada6/bindhack.c' -P /tmp/

compile it

gcc -fPIC -static -shared -o /tmp/bindhack.so /tmp/bindhack.c -lc -ldl

Copy it to library folder

cp /tmp/bindhack.so /usr/lib/ && chmod +x /usr/lib/bindhack.so

Optional (ignore if you have it already )

echo 'nameserver 8.8.8.8' >> /etc/resolv.conf

using bindhack

BIND_ADDR=<source ip> LD_PRELOAD=/usr/lib/bindhack.so <command here>

Example

 

you can add below function in your .bashrc to spin it at any time

 

 

take a look at bindhack.c

 

 

Speed up Ansible

Update to the latest version. Ansible 2.0 is slower than Ansible 1.9 because it included an important change to the execution engine to allow any user to choose the execution algorithm to be used. In the versions that followed, and mostly in 2.1, big optimizations have been done to increase execution speed, so be sure to be running the latest possible version.

Profiling Tasks

The best way I’ve found to time the execution of Ansible playbooks is by enabling the profile_tasks callback. This callback is included with Ansible and all you need to do to enable it is add callback_whitelist = profile_tasks to the [defaults] section of your ansible.cfg:
# ansible.cfg

 

Enable pipelining

You can enable pipelining by simply adding pipelining = True to the [ssh_connection]area of your ansible.cfg or by by using the ANSIBLE_PIPELINING and ANSIBLE_SSH_PIPELINING environment variables.
# ansible.cfg
You’ll also need to make sure that requiretty is disabled in /etc/sudoers on the remote host, or become won’t work with pipelining enabled.

Enable Mitogen for Ansible

Enabling Mitogen for Ansible is as simple as downloading and extracting the plugin, then adding 2 lines to the [defaults] section of your ansible.cfg:
# ansible.cfg

SSH multiplexing

The first thing to check is whether SSH multiplexing is enabled and used. This gives a tremendous speed boost because Ansible can reuse opened SSH sessions instead of negotiating new one (actually more than one) for every task. Ansible has this setting turned on by default. It can be set in configuration file as follows:

But be careful to override  ssh_args  — if you don’t set ControlMaster   and ControlPersist  while overriding, Ansible will “forget” to use them.

To check whether SSH multiplexing is used, start Ansible with  -vvvv  option:
ansible test -vvvv -m ping

UseDNS

UseDNS is an SSH-server setting (/etc/ssh/sshd_config file) which forces a server to check a client’s PTR-record upon connection. It may cause connection delays especially with slow DNS servers on the server side. In modern Linux distribution, this setting is turned off by default, which is correct.

PreferredAuthentications

It is an SSH-client setting which informs server about preferred authentication methods. By default Ansible uses:
-o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey
So if GSSAPI Authentication is enabled on the server (at the time of writing this it is turned on in RHEL EC2 AMI) it will be tried as the first option, forcing the client and server to make PTR-record lookups. But in most cases, we want to use only public key auth. We can force Ansible to do so by changing ansible.cfg:

 

Facts Gathering

At the start of playbook execution, Ansible collects facts about remote system (this is default behaviour for ansible-playbook but not relevant to ansible ad-hoc commands). It is similar to calling “setup” module thus requires another ssh communication step. If you don’t need any facts in your playbook (e.g. our test playbook) you can disable fact gathering:

Fork

Until this moment we discussed how to speed up playbook execution on a given remote host. But if you run playbook against tens or hundreds of hosts, Ansible internal performance becomes a bottleneck. For example, there’s preconfigured number of forks – number of hosts that can be interacted simultaneously. You can change this value in  ansible.cfg file:

 

The default value is 5, which is quite conservative. You can experiment with this setting depending on your local CPU and network bandwidth resources.
Another thing about forks is that if you have a lot of servers to work with and a low number of available forks, your master ssh-sessions may expire between tasks. Ansible uses linear strategy by default, which executes one task for every host and then proceeds to the next task. This way if time between task execution on the first server and on the last one is greater than ControlPersist then master socket will expire by the time Ansible starts execution of the following task on the first server, thus new ssh connection will be required.

Poll Interval

When module is executed on remote host, Ansible starts to poll for its result. The lower is interval between poll attempts, the higher is CPU load on Ansible control host. But we want to have CPU available for greater forks number (see above). You can tweak poll interval in  ansible.cfg:

 

If you run “slow” jobs (like backups) on multiple hosts, you may want to increase the interval to 0.05   to use less CPU.
Hope this helps you to speed up your setup. Seems like there are no more items in environment check-list and further speed gains only possible by optimizing your playbook code.

Asynchronous Actions and Polling

By default tasks in playbooks block, meaning the connections stay open until the task is done on each node. This may not always be desirable, or you may be running operations that take longer than the SSH timeout.
To avoid blocking or timeout issues, you can use asynchronous mode to run all of your tasks at once and then poll until they are done.
The behaviour of asynchronous mode depends on the value of poll.

Avoid connection timeouts: poll > 0

When poll is a positive value, the playbook will still block on the task until it either completes, fails or times out.
In this case, however, async explicitly sets the timeout you wish to apply to this task rather than being limited by the connection method timeout.
To launch a task asynchronously, specify its maximum runtime and how frequently you would like to poll for status. The default poll value is 15 seconds if you do not specify a value for poll:

 

Concurrent tasks: poll = 0

When poll is 0, Ansible will start the task and immediately move on to the next one without waiting for a result.
From the point of view of sequencing this is asynchronous programming: tasks may now run concurrently.
The playbook run will end without checking back on async tasks.
The async tasks will run until they either complete, fail or timeout according to their async value.
If you need a synchronization point with a task, register it to obtain its job ID and use the async_status module to observe it.
You may run a task asynchronously by specifying a poll value of 0:

 

Enable fact_caching

By enabling this value we’re telling Ansible to keep the facts it gathers in a local file. You can also set this to a redis cache. See the documentation for details.
Fact_caching is what happens when Ansible says, “Gathering facts” about your target hosts. If we don’t change our targets hardware (or virtual hardware) very often this can be very helpful. Enable it by adding this to your ansible.cfg file:
Enable facts caching mechanism
If you still need some of the facts groups, but at the same time the gathering process is still slow for you, you could try use fact caching.
Caching enables Ansible to cache the facts for a given host in some kind of backend.
Currently the caching plugin supports the following cache backend:

  •  
More information on the caching plugin, could be found here:
This is an example configuration of facts caching in json files

References:

1.https://dzone.com/articles/speed-up-ansible

2.https://habr.com/en/post/453446/

3.https://www.toptechskills.com/ansible-tutorials-courses/speed-up-ansible-playbooks-pipelining-mitogen/

4.https://www.youtube.com/watch?v=NZUYAbGs-ec