Monthly Archives: June 2021

How to run twisted script as daemon without twistd command

Warning:- this article assumes you are familiar with twisted
we have scenarios like run different services on different ports
to create multiservice architecture.
So, here we cannot run the services on from the command line
we have to run them as daemon service, like using cron jobs in linux.

– Create a simple twisted with protocol and factory and save to test.py or any name you like.

#!/usr/bin/env python2.7

from twisted.internet import protocol, reactor

class echo(protocol.Protocol):
        def dataReceived(self, data):
                self.transport.write(data.upper())

class echoFactory(protocol.Factory):
        def buildProtocol(sef,addr):
                print 'called'
                return echo()

reactor.listenTCP(8000,echoFactory())
reactor.run()

this is a normal echo server script which can run using

./test.py # assuming the script is saved in test.py

But our script cannot run this is as a daemon service
daemon service is some task runs in the background, and it logs to some specific location, not on stdout. When we run this script like specified above, it blocks the current terminal ( if you are running from a terminal ) which is not something daemon script do.
To make this script run as a daemon, we need to use twistd inside the script
means it’ll run and execute, create a process in the background.

from twisted.scripts import twistd
import sys
sys.argv.append('-y dummy')
sys.argv.append('--pidfile={0} --logfile=/dev/null'.format('/tmp/echo.pid'))

application = service.Application('echo_daemon')
tcp_service = internet.TCPServer(interface='127.0.0.1',port=8000, factory=factory)
tcp_service.setServiceParent(application)

class ApplicationRunner(twistd._SomeApplicationRunner):
    def createOrGetApplication(self):
        return application

    def run(self):
        self.preApplication()
        self.application = self.createOrGetApplication()
        self.postApplication()


twistd._SomeApplicationRunner = ApplicationRunner
twistd.run()

#~ python2.7 test.py

How should we cross-check if the service is running or not
Use the following command on Linux system (in shell/terminal for simple word).

netstat -ntulp

You get a list of all ports (UDP, TCP) open on the system.

How to edit file with sed

This tutorial is about how to replace the pattern with sed command in the same file you are reading from.

What is sed?
Sed stands for stream editor, as per the Linux man page.
But sed is more than it. We can write more powerful bash scripts with just a single line using sed. It can be used for fetching specific lines, a bunch of lines, replace the character with patterns and so much. Basically, just manipulate the stream of characters with whatever logic you projectile

Let’s get started,
We understand more with the task in hand. So, our task for this tutorial is to replace some characters with our characters.
Suppose we have a file like content below,

  304  PULSEaudio -k
  306  PULSEaudio --cleanup-shm 
  310  PULSEaudio --check 
  311  PULSEaudio --start 
  312  PULSEaudio --kill 
  322  PULSEaudio -k
  323  PULSEaudio --check 
  324  killall PULSEaudio
  325  PULSEaudio --check 
  331  ls * | grep -e PULSE
  332  cd PULSE/
  340  PULSEaudio -k
  344  PULSEaudio -D
  345  PULSEaudio -d
  346  service PULSEaudio status
  348  ps -eo "user args" | grep PULSE
  350  ps -eo "user args" | grep PULSE
  351  PULSEaudio -k
  352  killall PULSEaudio 

And we are supposed to replace the characters PULSE to pulse.
Either, we can open this file in VIM and type command like

 %s/PULSE/pulse/gc

If you are familiar with vim. You’ll know what I am talking about
But if you are using this output for some reason in your bash script, you need to do this with a single command.

Here comes our savior sed.

sed s/pattern/replace_char/ <file_name>

This command does our task, but the problem is it gives output on stdout.

Common error:- we generally try to redirect that output to the file we are editing
If we are editing the file named replace.txt then the command will be
sed s/PULSE/pulse/ replace.txt 2> replace.txt

But sed creates a problem here. It doesn’t work that way it’s not sed’s problem, Its problem with the order of file descriptors it set.

This is a common error, we want to modify a file using something that reads from a file and writes the result to stdout. To do this, we redirect stdout to the file we want to modify. The problem here is that, as we have seen, the redirections are setup before the command is actually executed.
So BEFORE sed starts, standard output has already been redirected, with the additional side effect that, because we used >, “file” gets truncated. When sed starts to read the file, it contains nothing
( if you don’t know what it is read this link https://wiki.bash-hackers.org/howto/redirection_tutorial ).

Sed added one feature which internally sets this redirection of a file descriptor for us. Use -i option to overcome this problem.

The final command will be

sed s/PULSE/pulse/ replace.txt -i

How to make hello world program in wxPython

In this article we will look at creating a simple hello world program using wxPython. This program will create and display simple window with a big button on it. Up on clicking the button program will exit. Use the following code to create hello world program. You must already have wxPython library installed

Hello world program

import wx

class MainWindow(wx.Frame):
    def __init__(self,parent):
        wx.Frame.__init__(self,parent,title="Hello World")
        self.killButton = wx.Button(self,label="Kill Me")
        self.killButton.Bind(wx.EVT_BUTTON,self.kill)
        self.Show()

    def kill(self,event):
        self.Close()
        print("Bye Bye cruel world")
        
app = wx.App(False)
frame = MainWindow(None)
app.MainLoop()

Explanation

We will go line by line here and try to explain what’s going on in this program. Most of the lines of self explanatory. If you are just getting started in programming the following explanation will be helpful.

  1. Import wxpython library
  2. Inherit from wx.Frame class. This is useful style for most of the programs that you will build. You can create one base frame or window and put rest of GUI widgets on top of it like Text controls, buttons,images, tables etc.
  3. Instantiate the inherited frame the desired title. parent argument is usually None for main windows.
  4. Create a button with label “Kill Me”. The first argument is parent. In this case we use “self” which is the main window we have just created.
  5. Bind the button click event (EVT_BUTTON) of the killButton to kill method. Whenever, EVT_BUTTON event is fired aka the killButton is clicked, kill method will be called.
  6. This line will cause the window to get displayed on screen. It’s customary to call this method after being done with construction of GUI i.e. create main window, place widgets, bind event like we did here.
  7. Create wxPython application by call wx.App. Every wxPython program must have this application.
  8. Start the main loop. Which will hand over control to wxPython library. This post explains why main loop has to be called.

Output

This program will launch the following window. The button takes all the available space on the window since there are no other widgets. You need a few more lines of code to make the button look like what users are used to – small and horizontal. You can exit the program by clicking the button.

How to install wxpython

In this post we will go over the topic of easy way to install wxpython. The following command will install wxpython easily in python3.

pip install wxpython

Python 2

Older versions of wxpython can be installed by downloading binaries (for windows) from sourceforge . The binaries are available for both Windows and MacOS.

Linux

The pip install command should work on linux also. However, if you are stuck on old linux vesions. You can try installing wxpython using your distro’s package manager like apt-get or yum. You can search for exact version and package name available on your platform using the following commands for debian variants.

apt-cache search wxpython

┌──(kali㉿kali)-[~]
└─$ apt-cache search wxpython
gnumed-client - medical practice management - Client
psychopy - environment for creating psychology stimuli in Python
pyscanfcs - scientific tool for perpendicular line scanning FCS
python3-genx - differential evolution algorithm for fitting
python3-opengl - Python bindings to OpenGL (Python 3)
python3-pyface - traits-capable windowing framework
python3-squaremap - wxPython control to display hierarchic data as nested squares
python3-wxgtk-media4.0 - Python 3 interface to the wxWidgets Cross-platform C++ GUI toolkit (wx.media)
python3-wxgtk-webview4.0 - Python 3 interface to the wxWidgets Cross-platform C++ GUI toolkit (wx.html2)
python3-wxgtk4.0 - Python 3 interface to the wxWidgets Cross-platform C++ GUI toolkit
soundgrain - Graphical interface to control granular sound synthesis modules
wxglade - GUI designer written in Python with wxPython
wxpython-tools - Tools from the wxPython distribution

Successful installation

Use the following command to check if the installation is successful or not.

PS C:\Users\godson> python
Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>> wx.version()
'4.1.1 msw (phoenix) wxWidgets 3.1.5'
>>>

As shown above it should print current version of wxpython installed if the installation is successful.

A more extensive tutorial is available here

mysql

How To Import and Export Databases in MySQL

MySQL is an open-source relational database management system. Its name is a combination of “My”, the name of co-founder Michael Widenius’s daughter, and “SQL”, the abbreviation for Structured Query Language.

A relational database organizes data into one or more data tables in which data types may be related to each other; these relations help structure the data. SQL is a language programmers use to create, modify and extract data from the relational database, as well as control user access to the database. In addition to relational databases and SQL, an RDBMS like MySQL works with an operating system to implement a relational database in a computer’s storage system, manages users, allows for network access and facilitates testing database integrity and creation of backups.

What is mysql?

mysql is a simple SQL shell (with GNU readline capabilities). It supports interactive and
non-interactive use. When used interactively, query results are presented in an ASCII-table format.
When used non-interactively (for example, as a filter), the result is presented in tab-separated
format. The output format can be changed using command options.

What is mysqldump?

The mysqldump client is a backup program originally written by Igor Romanenko. It can be used to dump
a database or a collection of databases for backup or transfer to another SQL server (not necessarily
a MariaDB server). The dump typically contains SQL statements to create the table, populate it, or
both. However, mysqldump can also be used to generate files in CSV, other delimited text, or XML
format.

Export a MySQL Database

Use mysqldump to export your database:

mysqldump -u username -p database_name > database_name-dump.sql

You can compress the data on the run using pipe and gzip.

mysqldump -u username -p database_name | gzip > database_name-dump.sql.gz

*Using GZIP will save a lot of space on disk for huge databases.

Import a MySQL Database

Use mysql to import your database:

Create the database first.

mysql > CREATE DATABASE database_name;

Import the database now.

mysql -u username -p database_name < database_name-dump.sql

If the file is compressed with gzip. use zcat to extract on the run.

zcat database_name-dump.sql.gz | mysql -u username -p database_name

Handy scripts for admins who do backups daily

bkpmysqlgz() {
    set -x
    DEST=${1:-/opt/backups}
    shift
    sudo mkdir -p $DEST
    DATE=$(DATE)
    if command -v pv > /dev/null 2>&1; then
        sudo mysqldump $@ | pv | gzip > $DEST/"${@: -1}"-$DATE.sql.gz
    else
        sudo mysqldump $@ | gzip > $DEST/"${@: -1}"-$DATE.sql.gz
    fi
    ls -lh $DEST/"${@: -1}"-$DATE.sql.gz
    set +x
}

using script.

$ bkpmysqlgz /opt/backups -u root -p secret dbname