Locking your bash script against parallel execution

Sometimes there’s a need to ensure only one copy of a script runs, i.e prevent two or more copies running simultaneously. Imagine an important cronjob doing something very important, which will fail or corrupt data if two copies of the called program were to run at the same time. To prevent this, a form of MUTEX (mutual exclusion) lock is needed.

The basic procedure is simple: The script checks if a specific condition (locking) is present at startup, if yes, it’s locked – the script doesn’t start.

This article describes locking with common UNIX® tools.

Method 1

setting the noclobber shell option (set -C). This will cause redirection to fail, if the file the redirection points to already exists (using diverse open() methods). Need to write a code example here.

 

Method 2

A simple way to get that is to create a lock directory – with the mkdir command. It will:

create a given directory only if it does not exist, and set a successful exit code
it will set an unsuccessful exit code if an error occurs – for example, if the directory specified already exists
With mkdir it seems, we have our two steps in one simple operation. A (very!) simple locking code might look like this:

In case mkdir reports an error, the script will exit at this point – the MUTEX did its job!

Apple iOS App Provisioning

A distribution certificate identifies your team/organization within a distribution App Provisioning profile and allows you to submit your app to the Apple App Store.

The workflow for developing and distributing iOS apps can be complex and difficult to understand. This article explain the steps needed to manage certificates and provisioning profiles and assist developer who are starting to develop in-house iOS apps.

A provisioning profile is a collection of digital entities that uniquely ties developers and devices to an authorized iPhone Development Team and enables a device to be used for testing.

The following steps describe the high level activities required to manage and distribute apps.

  1. Manage Certificates for development and production
    1. Create certificate for Development
    2. Create certificate for Production
  2. Register AppId and DeviceId
    1. Add Device ID’s
    2. Register AppID’s
  3. Create App Provisioning Profiles for Project
    1. Create Provisioning Profile for Developement & Production
    2. Apply AppID to Provisioning Profile
    3. Apply Certificate to App Provisioning Profile
    4. Download Profile and add it in Xcode

Step 1: Login

Go to https://developer.apple.com and click on Account (you must have an Apple Developer account to begin)

1. Click Log In, choose Select Certificates, Identifiers & Profiles

Step 2 : create Certificate

On the left menu select Certificates

1. Select add button “+” at the top right to create a new Certificate

2. Select “iOS Distribution (App Store and Ad Hoc)” and press Continue

3. Developers will need to generate a Certificate Signing Request (CSR) from their keychain and perform the Request Certificate function. Then select Signing certificate and Generate and Download certificate.

4. Click on Downloaded Certificate, it will added to KeyChain.

Step 3 : RegisterAppID

  1. From the left menu select Identifiers -> Click on Add new -> Select AppID’s
  2. Describe Name, and paste your project’s BundleID and select Capability which your app provides
  3. Click On Register Button. Now your AppID is Registered.

Step 4 : Create App Provisioning Profile

On the left tab under Provisioning Profiles, select Distribution

1. Select add button “+” at the top right to create a new profile

2. Select “App Store” and press Continue

iOS Provisioning Profile

 

 

3. Select App ID and press Continue

 

iOS Provisioning Profile

 

 

4. Select the Certificate you wish to include in this provisioning profile (the certificate the app was signed with) and click Continue. Next, select the devices you wish to include in the provisioning profile. The certificate is a public/private key-pair, which identifies who developed the app.

 

5. Create a name for your profile and click Generate. You might want to include “Distribution” in the name so you can distinguish this one from testing.

6. Download Your Profile and by clicking it will be added to Xcode.

 

Howto reverse proxy in nginx

Proxying is typically used to distribute the load among several servers, seamlessly show content from different websites, or pass requests for processing to application servers over protocols other than HTTP.

When NGINX proxies a request, it sends the request to a specified proxied server, fetches the response, and sends it back to the client. It is possible to proxy requests to an HTTP server (another NGINX server or any other server) or a non-HTTP server (which can run an application developed with a specific framework, such as PHP or Python) using a specified protocol.

1. To pass a request to an HTTP proxied server, the proxy_pass directive is specified inside a location. For example:

 2. This address can be specified as a domain name or an IP address. The address may also include a port:

3. To pass a request to a non-HTTP proxied server, the appropriate **_pass directive should be used:

  • fastcgi_pass passes a request to a FastCGI server
  • uwsgi_pass passes a request to a uwsgi server
  • scgi_pass passes a request to an SCGI server
  • memcached_pass passes a request to a memcached server

4. Passing Request Headers

 

5. To disable buffering in a specific location, place the proxy_buffering directive in the location with the off parameter, as follows:

 

 

WSL vs WSL 2 – performance

WSL 2 is a new version of the architecture that powers the Windows Subsystem for Linux to run ELF64 Linux binaries on Windows. Its primary goals are to increase file system performance, as well as adding full system call compatibility. This new architecture changes how these Linux binaries interact with Windows and your computer’s hardware, but still provides the same user experience as in WSL 1 (the current widely available version). Individual Linux distros can be run either as a WSL 1 distro, or as a WSL 2 distro, can be upgraded or downgraded at any time, and you can run WSL 1 and WSL 2 distros side by side. WSL 2 uses an entirely new architecture that uses a real Linux kernel.

It’s a major reworking of the original WSL concept, moving away from translating Linux system calls to Windows to shipping a complete Linux kernel that runs alongside Windows’ own kernel.

The reasons for doing this are many, but the main one is simple: It’s impossible for an emulator that ships twice a year to keep up with the changes in the Linux kernel, changes that Linux binaries depend on. If Windows is to support developers building Linux apps for the cloud, then it needs to be more than consistent, it needs to be compatible.

 

Linux kernel in WSL 2

The Linux kernel in WSL 2 is built in house from the latest stable branch, based on the source available at kernel.org. This kernel has been specially tuned for WSL 2. It has been optimized for size and performance to give an amazing Linux experience on Windows and will be serviced through Windows updates, which means you will get the latest security fixes and kernel improvements without needing to manage it yourself.

Increased file IO performance

File intensive operations like git clone, npm install, apt update, apt upgrade, and more will all be noticeably faster. The actual speed increase will depend on which app you’re running and how it is interacting with the file system. Initial versions of WSL 2 run up to 20x faster compared to WSL 1 when unpacking a zipped tarball, and around 2-5x faster when using git clone, npm install and cmake on various projects.

Sockets performance benchmarks

WSL

wsl

 

WSL 2

wsl2

The Ubuntu 18.04 LTS WSL instance was used for testing with its default packages. In addition to looking at the WSL1 vs. WSL2 performance of Ubuntu 18.04, Ubuntu 18.04.2 LTS itself was also tested bare metal on the same system for looking at the raw performance of Ubuntu on the Intel desktop being tested.

Full System Call Compatibility

Linux binaries use system calls to perform many functions such as accessing files, requesting memory, creating processes, and more. In WSL 1 we created a translation layer that interprets many of these system calls and allows them to work on the Windows NT kernel. However, it’s challenging to implement all of these system calls, resulting in some apps being unable to run in WSL 1. Now that WSL 2 includes its own Linux kernel it has full system call compatibility. This introduces a whole new set of apps that you can run inside of WSL. Some exciting examples are the Linux version of Docker, as well as FUSE!

Using WSL 2 means you can also get the most recent improvements to the Linux kernel much faster than in WSL 1, as we can simply update the WSL 2 kernel rather than needing to reimplement the changes ourselves.

WSL 2 will be a much more powerful platform for you to run your Linux apps on and will empower you to do more with a Linux environment on Windows.

 

How to fix different times in Dual boot mode ( Windows and Linux)

Your PC stores the time in a hardware clock on its motherboard. The clock keeps track of time, even when the computer is off. By default, Windows assumes the time is stored in local time, while Linux assumes the time is stored in UTC time and applies an offset. This leads to one of your operating systems showing the wrong time in a dual boot situation.

To fix this, you have two options: Disable RTC in Linux, or make Windows use UTC time. Don’t follow both steps of instructions or they still won’t be speaking the same language! We recommend you make Linux use local time, if possible.

1. Disable RTC on Linux

timedatectl set-local-rtc 1 --adjust-system-clock

                                        OR

2. Use UTC in windows

How to use ipset command on linux to block bulk IPs

ipset is a companion application for the iptables Linux firewall. It allows you to setup rules to quickly and easily block a set of IP addresses, among other things.

Installation

Debian based system

# apt install ipset

Redhat based system

# yum install ipset

Blocking a list of network

Start by creating a new “set” of network addresses. This creates a new “hash” set of “net” network addresses named “myset”.

or

Add any IP address that you’d like to block to the set.

Finally, configure iptables to block any address in that set. This command will add a rule to the top of the “INPUT” chain to “-m” match the set named “myset” from ipset (–match-set) when it’s a “src” packet and “DROP”, or block, it.

Blocking a list of IP addresses

Start by creating a new “set” of ip addresses. This creates a new “hash” set of “ip” addresses named “myset-ip”.

or

Add any IP address that you’d like to block to the set.

Finally, configure iptables to block any address in that set.

Making ipset persistent

The ipset you have created is stored in memory and will be gone after reboot. To make the ipset persistent you have to do the followings:

First save the ipset to /etc/ipset.conf:

Then enable ipset.service, which works similarly to iptables.service for restoring iptables rules.

Other Commands

To view the sets:

or

To delete a set named “myset”:

or

To delete all sets:

How to Debug the Execution of a Program in Linux

strace is a useful diagnostic, instructional, and debugging tool. System administrators, diagnosticians and trouble-shooters will find it invaluable for solving problems with programs for which the source is not readily available since they do not need to be recompiled in order to trace them. Students, hackers and the overly-curious will find that a great deal can be learned about a system and its system calls by tracing even ordinary programs. And programmers will find that since system calls and signals are events that happen at the user/kernel interface, a close examination of this boundary is very useful for bug isolation, sanity checking and attempting to capture race conditions.

Trace the Execution

You can use strace command to trace the execution of any executable. The following example shows the output of strace for the Linux uname command.

Counting number of syscalls

Run the ls command counting the number of times each system call was made and print totals showing the number and time spent in each call (useful for basic profiling or bottleneck isolation):

Save the Trace Execution to a File Using Option -o

The following examples stores the strace output to output.txt file.

Print Timestamp for Each Trace Output Line Using Option -t

To print the timestamp for each strace output line, use the option -t as shown below.

Tracing only network related system calls

Trace just the network related system calls of ping command

 

Viewing files opened by a process/daemon using tracefile

tracefile: Output from cmd on stdout can mess up output from strace.

Notes

It is a pity that so much tracing clutter is produced by systems employing shared libraries.

It is instructive to think about system call inputs and outputs as data-flow across the user/kernel boundary. Because user-space and kernel-space are separate and address-protected, it is sometimes possible to make deductive inferences about process behavior using inputs and outputs as propositions.

In some cases, a system call will differ from the documented behavior or have a different name. For example, on System V-derived systems the true time(2) system call does not take an argument and the stat function is called xstat and takes an extra leading argument. These discrepancies are normal but idiosyncratic characteristics of the system call interface and are accounted for by C library wrapper functions.

On some platforms a process that has a system call trace applied to it with the -p option will receive a SIGSTOP . This signal may interrupt a system call that is not restartable. This may have an unpredictable effect on the process if the process takes no action to restart the system call.

 

Network namespaces – part 1

Linux namespaces are a relatively new kernel feature which is essential for implementation of containers. A namespace wraps a global system resource into an abstraction which will be bound only to processes within the namespace, providing resource isolation. In this article I discuss network namespace and show a practical example.

What is namespace?

A namespace is a way of scoping a particular set of identifiers. Using a namespace, you can use the same identifier multiple times in different namespaces. You can also restrict an identifier set visible to particular processes.

For example, Linux provides namespaces for networking and processes, among other things. If a process is running within a process namespace, it can only see and communicate with other processes in the same namespace. So, if a shell in a particular process namespace ran ps waux, it would only show the other processes in the same namespace.

Linux network namespaces

In a network namespace, the scoped ‘identifiers’ are network devices; so a given network device, such as eth0, exists in a particular namespace. Linux starts up with a default network namespace, so if your operating system does not do anything special, that is where all the network devices will be located. But it is also possible to create further non-default namespaces, and create new devices in those namespaces, or to move an existing device from one namespace to another.

Each network namespace also has its own routing table, and in fact this is the main reason for namespaces to exist. A routing table is keyed by destination IP address, so network namespaces are what you need if you want the same destination IP address to mean different things at different times – which is something that OpenStack Networking requires for its feature of providing overlapping IP addresses in different virtual networks.

Each network namespace also has its own set of iptables (for both IPv4 and IPv6). So, you can apply different security to flows with the same IP addressing in different namespaces, as well as different routing.

Any given Linux process runs in a particular network namespace. By default this is inherited from its parent process, but a process with the right capabilities can switch itself into a different namespace; in practice this is mostly done using the ip netns exec NETNS COMMAND… invocation, which starts COMMAND running in the namespace named NETNS. Suppose such a process sends out a message to IP address A.B.C.D, the effect of the namespace is that A.B.C.D will be looked up in that namespace’s routing table, and that will determine the network device that the message is transmitted through.

Lets play with ip namespaces

By convention a named network namespace is an object at /var/run/netns/NAME that can be opened. The file descriptor resulting from opening /var/run/netns/NAME refers to the specified network namespace.

create a namespace

power up loopback device

open up a namespace shell

now we can use this shell like user shell where it uses ns1 namespace only

 

In part-2  , I will explain how to connect to internet from ns1 namespace and adding custom routes.

Setup Xamarin Environment on Mac & Visual Studio

Below I have explained how to setup Xamarin environment on mac operating system step by step.

1. Download Visual studio : 

      Download Visual Studio with below link

     https://visualstudio.microsoft.com/downloads/

      

At Microsoft website, you will have three options of  Visual Studio edition to choose from. Choose one according to your need. To download Visual Studio just click on download button and an installer .dmg file will be downloaded.

2. Install Visual Studio:

   Click on downloaded dmg file and below screen will be presented

    

Select from the different Platforms you need to develop apps for on Xamarin and press the Install button. Once Visual Studio installation is complete, we need to setup environment for both Android and Apple.

3. Setup Android SDK:

    To setup Android SDK open Visual Studio and go to :-

    Tools -> SDK Manager ->Android -> Locations

    

Set path for SDK ,NDK and JDK to your local machine locations. Once correct  path is given a green tick will appear on right.This completes our Android SDK   setup.

4. Apple Setup (for both iOS and Mac apps development):

You need latest Xcode to setup Apple environment. If you have Xcode preinstalled on your machine then it automatically configures and we don’t  have to do anything. If you are installing Xcode after installation of Visual  Studio then follow below steps to setup.

a. Download latest Xcode from apple store and Install it on your machine.

b. Go to Tool -> SDK Manager -> Apple

 

Give path to your Xcode.app . You will see green check mark once the correct path is given. This completes Apple environment setup.

That is all.  Now you can start your Android and iOS development on Xamarin. Happy Coding!

How to create Gridview using Recylerview Android

First let’s understand what Gridview and Recylerview are, in Android.

Gridview

A view that shows items in two-dimensional scrolling grid is known as Gridview. GridView layout in one of the most useful layouts in android to create a scrolling grid (rows & columns).

Recylerview

Recylerview is introduced in Android 5.0 Lollipop. The Recylerview widget is a more advanced and flexible version of Listview. It is a container used to display a large number of data sets that can be scrolled very efficiently by maintaining a limited number of views.

Now let’s start implementing Gridview

First, we need to add below dependency in build.gradle file at app level module.

After that, we need to add Recylerview widget in your main XML file.

Now we need to create item_logo.xml for Gridview row item.

We need to create Adapter Object. An adapter in Android carries the data from a source (e.g. List<> ) and delivers it to a layout (.xml file).  The Adapter provides access to the data items.

To display images we can use Glide dependency.

Now we need to set data into Adapter.

GridLayoutManager is a Recylerview Layout Manager implementation to lay out items in a grid.

In the above code “3” is a number of columns in per row.

Output

 

 

 

 

 

 

 

 

 

 

That’s it, Happy Coding 🙂

Reference:-  https://developer.android.com/guide/topics/ui/layout/recyclerview