How to truncate empty lines using sed

Sed command is very helpful  for text processing. It’s just matter of regular expression. Here is how you can  truncate empty lines  in the file using command sed.

The above command will give you the output by deleting empty lines. If you want to modify the original file itself. You use the option -i

Example:

I have a file with following data and named as numbers

Now see, the way to truncate empty lines using sed

The above command won’t change the original file. If you want to change original use option -i

Explanation:

Sed interprets ‘/^s*$/d’  and giving us the desired output. Here  text between slashes(/regular expression/) represent the regular expression followed by sed command “d”  means delete.  So command sed matches the lines with given regular expression. If it finds the match it applies the command.

 

How to shuffle lines in the file in linux

We can shuffle lines in the file in linux using following commands

  • shuf
  • sed and sort
  • awk
  • python

As an example we will take a file shuffle_mylines.txt  having numbers till 10 each digit in a new line.

Create a file using following command

$ seq 10  > shuffle_mylines.txt

Command shuf

This command is light wight and straight forward. You just need to call this command with file name as an argument.

Shuffle lines using sed

You may have already know about command sed(Stream Editor). It is one of the command widely used for text processing in unix/linux. We can’t shuffle line using single sed command, but we will do by combining other commands. Let’s take a look at following command,

How does it work?

Breakdown of above command,

Commands we have used in the above example are,

  • cat
  • while loop
  • $RANDOM   environment variable
  • soft
  • tail
  • sed

 

Now, lets come to see how this command work. First command cat will read the file content and will pipe it to shell while loop

Where, while loop will read the piped input into variable x and will iterate over all lines to generate  output  <random_number>:<line> as you can see $RANDOM:$x. Where $RANDOM is the environment variable, each time you query this variable you will get random number. Which is useful for to shuffle lines.

Then, we will sort output of above while loop using sort command

Out put of this command will always be randomly shuffled lines. It’s because $RANDOM.

Output here would look like,

To remove preceded random values we will use sed.

That’s it. On every execution of this command you will get shuffled lines. You can redirect output to new file if you want to store using (>) or (>>).

 

Shuffle lines using awk

The awk is the programming language which is specially designed for text processing. We will use it to shuffle lines.

 

Another example using awk. It’s is similar to sed and sort example.

Shuffle lines in file using python

Python is popular scripting language widely used today from big projects to small scripts. We will see, how you can shuffle lines using python.

Python Example 1

In this example, we are passing file name as command line argument. Reading it and shuffling the lines of file and printing them on terminal.

The output can be redirected to  a file using redirect operator (> or >>)

Conclusion:

If you are looking for a quick shuffle command shuf is best choice or you can have a fun of using other ways to shuffle lines in the file.