Locking your bash script against parallel execution

Sometimes there’s a need to ensure only one copy of a script runs, i.e prevent two or more copies running simultaneously. Imagine an important cronjob doing something very important, which will fail or corrupt data if two copies of the called program were to run at the same time. To prevent this, a form of MUTEX (mutual exclusion) lock is needed.

The basic procedure is simple: The script checks if a specific condition (locking) is present at startup, if yes, it’s locked – the script doesn’t start.

This article describes locking with common UNIX® tools.

Method 1

setting the noclobber shell option (set -C). This will cause redirection to fail, if the file the redirection points to already exists (using diverse open() methods). Need to write a code example here.

 

Method 2

A simple way to get that is to create a lock directory – with the mkdir command. It will:

create a given directory only if it does not exist, and set a successful exit code
it will set an unsuccessful exit code if an error occurs – for example, if the directory specified already exists
With mkdir it seems, we have our two steps in one simple operation. A (very!) simple locking code might look like this:

In case mkdir reports an error, the script will exit at this point – the MUTEX did its job!

Apple iOS App Provisioning

A distribution certificate identifies your team/organization within a distribution App Provisioning profile and allows you to submit your app to the Apple App Store.

The workflow for developing and distributing iOS apps can be complex and difficult to understand. This article explain the steps needed to manage certificates and provisioning profiles and assist developer who are starting to develop in-house iOS apps.

A provisioning profile is a collection of digital entities that uniquely ties developers and devices to an authorized iPhone Development Team and enables a device to be used for testing.

The following steps describe the high level activities required to manage and distribute apps.

  1. Manage Certificates for development and production
    1. Create certificate for Development
    2. Create certificate for Production
  2. Register AppId and DeviceId
    1. Add Device ID’s
    2. Register AppID’s
  3. Create App Provisioning Profiles for Project
    1. Create Provisioning Profile for Developement & Production
    2. Apply AppID to Provisioning Profile
    3. Apply Certificate to App Provisioning Profile
    4. Download Profile and add it in Xcode

Step 1: Login

Go to https://developer.apple.com and click on Account (you must have an Apple Developer account to begin)

1. Click Log In, choose Select Certificates, Identifiers & Profiles

Step 2 : create Certificate

On the left menu select Certificates

1. Select add button “+” at the top right to create a new Certificate

2. Select “iOS Distribution (App Store and Ad Hoc)” and press Continue

3. Developers will need to generate a Certificate Signing Request (CSR) from their keychain and perform the Request Certificate function. Then select Signing certificate and Generate and Download certificate.

4. Click on Downloaded Certificate, it will added to KeyChain.

Step 3 : RegisterAppID

  1. From the left menu select Identifiers -> Click on Add new -> Select AppID’s
  2. Describe Name, and paste your project’s BundleID and select Capability which your app provides
  3. Click On Register Button. Now your AppID is Registered.

Step 4 : Create App Provisioning Profile

On the left tab under Provisioning Profiles, select Distribution

1. Select add button “+” at the top right to create a new profile

2. Select “App Store” and press Continue

iOS Provisioning Profile

 

 

3. Select App ID and press Continue

 

iOS Provisioning Profile

 

 

4. Select the Certificate you wish to include in this provisioning profile (the certificate the app was signed with) and click Continue. Next, select the devices you wish to include in the provisioning profile. The certificate is a public/private key-pair, which identifies who developed the app.

 

5. Create a name for your profile and click Generate. You might want to include “Distribution” in the name so you can distinguish this one from testing.

6. Download Your Profile and by clicking it will be added to Xcode.

 

Matrix Javascript SDK

Matrix Client-Server r0 SDK for JavaScript. This SDK can be run in a browser or in Node.js.

Quickstart

In a browser

Download either the full or minified version from https://github.com/matrix-org/matrix-js-sdk/releases/latest and add that as a <script> to your page. There will be a global variable matrixcs attached to window through which you can access the SDK. See below for how to include libolm to enable end-to-end-encryption.

Please check the working browser example for more information.

In Node.js

Ensure you have the latest LTS version of Node.js installed.

Using yarn instead of npm is recommended. Please see the Yarn install guide if you do not have it already.

yarn add matrix-js-sdk

See below for how to include libolm to enable end-to-end-encryption. Please check the Node.js terminal app for a more complex example.

To start the client:

You can perform a call to /sync to get the current state of the client:

To send a message:

To listen for message events:

By default, the matrix-js-sdk client uses the MemoryStore to store events as they are received. For example to iterate through the currently stored timeline for a room:

What does this SDK do?

This SDK provides a full object model around the Matrix Client-Server API and emits events for incoming data and state changes. Aside from wrapping the HTTP API, it:

  • Handles syncing (via /initialSync and /events)
  • Handles the generation of “friendly” room and member names.
  • Handles historical RoomMember information (e.g. display names).
  • Manages room member state across multiple events (e.g. it handles typing, power levels and membership changes).
  • Exposes high-level objects like Rooms, RoomState, RoomMembers and Users which can be listened to for things like name changes, new messages, membership changes, presence changes, and more.
  • Handle “local echo” of messages sent using the SDK. This means that messages that have just been sent will appear in the timeline as ‘sending’, until it completes. This is beneficial because it prevents there being a gap between hitting the send button and having the “remote echo” arrive.
  • Mark messages which failed to send as not sent.
  • Automatically retry requests to send messages due to network errors.
  • Automatically retry requests to send messages due to rate limiting errors.
  • Handle queueing of messages.
  • Handles pagination.
  • Handle assigning push actions for events.
  • Handles room initial sync on accepting invites.
  • Handles WebRTC calling.

Later versions of the SDK will:

  • Expose a RoomSummary which would be suitable for a recents page.
  • Provide different pluggable storage layers (e.g. local storage, database-backed)

 Usage

 Conventions

 Emitted events

The SDK will emit events using an EventEmitter. It also emits object models (e.g. Rooms, RoomMembers) when they are updated.

Promises and Callbacks

Most of the methods in the SDK are asynchronous: they do not directly return a result, but instead return a Promise which will be fulfilled in the future.

The typical usage is something like:

Alternatively, if you have a Node.js-style callback(err, result) function, you can pass the result of the promise into it with something like:

The main thing to note is that it is an error to discard the result of a promise-returning function, as that will cause exceptions to go unobserved. If you have nothing better to do with the result, just call .done() on it. See http://documentup.com/kriskowal/q/#the-end for more information.

Methods which return a promise show this in their documentation.

Many methods in the SDK support both Node.js-style callbacks and Promises, via an optional callback argument. The callback support is now deprecated: new methods do not include a callback argument, and in the future it may be removed from existing methods.

Examples

This section provides some useful code snippets which demonstrate the core functionality of the SDK. These examples assume the SDK is setup like this:

Automatically join rooms when invited

Print out messages for all rooms

Output:

Print out membership lists whenever they are changed

Output:

API Reference

A hosted reference can be found at http://matrix-org.github.io/matrix-js-sdk/index.html

This SDK uses JSDoc3 style comments. You can manually build and host the API reference from the source files like this:

Then visit http://localhost:8005 to see the API docs.

End-to-end encryption support

The SDK supports end-to-end encryption via the Olm and Megolm protocols, using libolm. It is left up to the application to make libolm available, via the Olm global.

It is also necessry to call matrixClient.initCrypto() after creating a new MatrixClient (but before calling matrixClient.startClient()) to initialise the crypto layer.

If the Olm global is not available, the SDK will show a warning, as shown below; initCrypto() will also fail.

If the crypto layer is not (successfully) initialised, the SDK will continue to work for unencrypted rooms, but it will not support the E2E parts of the Matrix specification.

To provide the Olm library in a browser application:

To provide the Olm library in a node.js application:

  • yarn add https://packages.matrix.org/npm/olm/olm-3.0.0.tgz (replace the URL with the latest version you want to use from https://packages.matrix.org/npm/olm/)
  • global.Olm = require('olm'); before loading matrix-js-sdk.

If you want to package Olm as dependency for your node.js application, you can use yarn add https://packages.matrix.org/npm/olm/olm-3.0.0.tgz. If your application also works without e2e crypto enabled, add --optional to mark it as an optional dependency.

Contributing

This section is for people who want to modify the SDK. If you just want to use this SDK, skip this section.

First, you need to pull in the right build tools:

Building

To build a browser version from scratch when developing::

To constantly do builds when files are modified (using watchify)::

To run tests (Jasmine)::

To run linting:

Python Matplotlib Library with Examples

What Is Python Matplotlib?

Matplotlib is a plotting library for the Python programming language and its numerical mathematics extension NumPy. It provides an object-oriented API for embedding plots into applications using general-purpose GUI toolkits like Tkinter, wxPython, Qt, or GTK+.

Pyplot is a Matplotlib module which provides a MATLAB-like interface. Matplotlib is designed to be as usable as MATLAB, with the ability to use Python and the advantage of being free and open-source. matplotlib.pyplot is a plotting library used for 2D graphics in the python programming language. It can be used in python scripts, shell, web application servers, and other graphical user interface toolkits.

There are several toolkits that are available that extend python Matplotlib functionality.

  • Basemap: It is a map plotting toolkit with various map projections, coastlines, and political boundaries.
  • Cartopy: It is a mapping library featuring object-oriented map projection definitions, and arbitrary point, line, polygon and image transformation capabilities.
  • Excel tools: Matplotlib provides utilities for exchanging data with Microsoft Excel.
    Mplot3d: It is used for 3-D plots.
  • Natgrid: It is an interface to the “natgrid” library for irregular gridding of the spaced data.
  • GTK tools: mpl_toolkits.gtktools provides some utilities for working with GTK. This toolkit ships with matplotlib, but requires pygtk.
  • Qt interface
  • Mplot3d: The mplot3d toolkit adds simple 3D plotting capabilities to matplotlib by supplying an axes object that can create a 2D projection of a 3D scene.
  • matplotlib2tikz: export to Pgfplots for smooth integration into LaTeX documents.

Types of Plots
There are various plots which can be created using python Matplotlib. Some of them are listed below:

  • Bar Graph
  • Histogram
  • Scatter Plot
  • Line Plot
  • 3D plot
  • Area Plot
  • Pie Plot
  • Image Plot

We will demonstrate some of them in detail.

But before that, let me show you elementary codes in python matplotlib in order to generate a simple graph.

So, with three lines of code, you can generate a basic graph using python matplotlib.

Let us see how can we add title, labels to our graph created by python matplotlib library to bring in more meaning to it. Consider the below example:

You can even try many styling techniques to create a better graph by changing the width or color of a particular line or what if you want to have some grid lines, there you need styling!

The style package adds support for easy-to-switch plotting “styles” with the same parameters as a matplotlibrc file.

There are a number of pre-defined styles provided by matplotlib. For example, there’s a pre-defined style called “ggplot”, which emulates the aesthetics of ggplot (a popular plotting package for R). To use this style, just add:

To list all available styles, use:

So, let me show you how to add style to a graph using python matplotlib. First, you need to import the style package from python matplotlib library and then use styling functions as shown in below code:

Now, we will understand the different kinds of plots. Let’s start with the bar graph!

Matplotlib: Bar Graph
A bar graph uses bars to compare data among different categories. It is well suited when you want to measure the changes over a period of time. It can be plotted vertically or horizontally. Also, the vital thing to keep in mind is that longer the bar, the greater is the value. Now, let us practically implement it using python matplotlib.

When I run this code, it generates a figure like below:


In the above plot, I have displayed a comparison between the distance covered by two cars BMW and Audi over a period of 5 days. Next, let us move on to another kind of plot using python matplotlib – Histogram

Matplotlib – Histogram
Let me first tell you the difference between a bar graph and a histogram. Histograms are used to show a graphical representation of the distribution of numerical data whereas a bar chart is used to compare different entities.

It is an estimate of the probability distribution of a continuous variable (quantitative variable) and was first introduced by Karl Pearson. It is a kind of bar graph.

To construct a histogram, the first step is to “bin” the range of values — that is, divide the entire range of values into a series of intervals — and then count how many values fall into each interval. The bins are usually specified as consecutive, non-overlapping intervals of a variable. The bins (intervals) must be adjacent and are often (but are not required to be) of equal size.

Basically, histograms are used to represent data given in the form of some groups or we can say when you have arrays or a very long list. X-axis is about bin ranges where Y-axis talks about frequency. So, if you want to represent the age-wise population in form of the graph then histogram suits well as it tells you how many exist in certain group range or bin if you talk in the context of histograms.

In the below code, I have created the bins in the interval of 10 which means the first bin contains elements from 0 to 9, then 10 to 19 and so on.

When I run this code, it generates a figure like below:

As you can see in the above plot, Y-axis tells about the age groups that appear with respect to the bins. Our biggest age group is between 40 and 50.

Matplotlib: Scatter Plot
A scatter plot is a type of plot that shows the data as a collection of points. The position of a point depends on its two-dimensional value, where each value is a position on either the horizontal or vertical dimension. Usually, we need scatter plots in order to compare variables, for example, how much one variable is affected by another variable to build a relation out of it.
Consider the below example:

As you can see in the above graph, I have plotted two scatter plots based on the inputs specified in the above code. The data is displayed as a collection of points having ‘high-income low salary’ and ‘low-income high salary.’

Scatter plot with groups
Data can be classified into several groups. The code below demonstrates:

The purpose of using “plt.figure()” is to create a figure object. It’s a Top-level container for all plot elements.

The whole figure is regarded as the figure object. It is necessary to explicitly use “plt.figure()”when we want to tweak the size of the figure and when we want to add multiple Axes objects in a single figure.

fig.add_subplot() is used to control the default spacing of the subplots.
For example, “111” means “1×1 grid, first subplot” and “234” means “2×3 grid, 4th subplot”.

You can easily understand by the following picture:

Next, let us understand the area plot or you can also say Stack plot using python matplotlib.

Matplotlib: Area Plot
Area plots are pretty much similar to the line plot. They are also known as stack plots. These plots can be used to display the evolution of the value of several groups on the same graphic. The values of each group are displayed on top of each other. It allows checking on the same figure the evolution of both the total of a numeric variable and the importance of each group.

A line chart forms the basis of an area plot, where the region between the axis and the line is represented by colors.

The above-represented graph shows how an area plot can be plotted for the present scenario. Each shaded area in the graph shows a particular bike with the frequency variations denoting the distance covered by the bike on different days. Next, let us move to our last yet most frequently used plot – Pie chart.

Matplotlib: Pie Chart
In a pie plot, statistical data can be represented in a circular graph where the circle is divided into portions i.e. slices of pie that denote a particular data, that is, each portion is proportional to different values in the data. This sort of plot can be mainly used in mass media and business.

In the above-represented pie plot, the bikes scenario is illustrated, and I have divided the circle into 4 sectors, each slice represents a particular bike and the percentage of distance traveled by it. Now, if you have noticed these slices adds up to 24 hrs, but the calculation of pie slices is done automatically for you. In this way, the pie charts are really useful as you don’t have to be the one who calculates the percentage of the slice of the pie.

Matplotlib: 3D Plot
Plotting of data along x, y, and z axes to enhance the display of data represents the 3-dimensional plotting. 3D plotting is an advanced plotting technique that gives us a better view of the data representation along the three axes of the graph.

Line Plot 3D

In the above-represented 3D graph, a line graph is illustrated in a 3-dimensional manner. We make use of a special library to plot 3D graphs which are given in the following syntax.
Syntax for plotting 3D graphs:

The import Axes3D is mainly used to create an axis by making use of the projection=3d keyword. This enables a 3-dimensional view of any data that can be written along with the above-mentioned code.

Surface Plot 3D

By default, it will be colored in shades of a solid color, but it also supports color mapping by supplying the cmap argument.

The rstride and cstride kwargs set the stride used to sample the input data to generate the graph. If 1k by 1k arrays are passed in, the default values for the strides will result in a 100×100 grid being plotted. Defaults to 10. Raises a ValueError if both stride and count kwargs (see next section) are provided.

Matplotlib: Image Plot

Matplotlib: Working With Multiple Plots
I have discussed multiple types of plots in python matplotlib such as bar plot, scatter plot, pie plot, area plot, etc. Now, let me show you how to handle multiple plots.

Setting Up Ansible for AWS with Dynamic Inventory (EC2)

If your Ansible inventory fluctuates over time, with hosts spinning up and shutting down in response to business demands, the static inventory solutions described in Working with Inventory will not serve your needs. You may need to track hosts from multiple sources

Ansible integrates all of these options via a dynamic external inventory system. Ansible supports two ways to connect with external inventory: Inventory Plugins and inventory scripts.

If you use Amazon Web Services EC2, maintaining an inventory file might not be the best approach, because hosts may come and go over time, be managed by external applications, or you might even be using AWS autoscaling. For this reason, you can use the EC2 external inventory script.

You can use this script in one of two ways.

  1. The easiest is to use Ansible’s -i command-line option and specify the path to the script after marking it executable:
  2. The second option is to copy the script to /etc/ansible/hosts and chmod +x it. You will also need to copy the ec2.ini file to /etc/ansible/ec2.ini. Then you can run ansible as you would normally.

You can test the script by itself to make sure your config is correct:


After a few moments, you should see your entire EC2 inventory across all regions in JSON.

If you use Boto profiles to manage multiple AWS accounts, you can pass --profile PROFILE name to the ec2.py script.

You can then run ec2.py --profile prod to get the inventory for the prod account, although this option is not supported by ansible-playbook. You can also use the AWS_PROFILE variable – for example:

 

ec2.py