Comandos Linux

Bash shuf command

Bash shuf command
Shuf is one of those commands that most bash programmers have not heard. For those who have, the experience itself is often eerie, like a whisper coming from a dark distant terminal where long-forgotten commands end up.  Those that venture further find shuf and are never the same again.

What is shuf?

shuf is a command-line utility like sort included in Coreutils. You may have guessed that it is used to pseudo randomize a given input in the same way you would shuffle a deck of cards. You guessed right!

Here we will cover the shuf command along with alternatives just in case you find yourself stuck without a shuf.

Where to find help on shuf

Once you know what shuf is, the next step is knowing how to use it. Like most command-line utilities, shuf comes with an -help long option.

Command

# shuf --help
Usage: shuf [OPTION]… [FILE]
or:  shuf -e [OPTION]… [ARG]…
or:  shuf -i LO-HI [OPTION]…
Write a random permutation of the input lines to standard output.
With no FILE, or when FILE is -, read standard input.
Mandatory arguments to long options are mandatory for short options too.
-e, --echo                treat each ARG as an input line
-i, --input-range=LO-HI   treat each number LO through HI as an input line
-n, --head-count=COUNT    output at most COUNT lines
-o, --output=FILE         write result to FILE instead of standard output
--random-source=FILE  get random bytes from FILE
-r, --repeat              output lines can be repeated
-z, --zero-terminated     line delimiter is NUL, not newline
--help     display this help and exit
--version  output version information and exit

The Three Ways of shuf

There are three ways to use the shuf command which are:

  1. file shuf
  2. list shuf
  3. range shuf

Each way has its advantages. Knowledge of all the ways beforehand may reduce the need to use other external commands in conjunction with the shuf command.

file shuf

File shuf is the most common way shuf is used in command line. When the -e or -i option are not included in the options, shuf will operate as file shuf. That is, the input to be shuffled will be a file whether standard input or any given file. The last word in the parameter list may be a filename. In case this parameter is omitted file is taken to be standard input from the shell or piped. A - may be included following the convention that standard input is inferred.

Here follow usage and ways to specify the file in file shuf.

Usage

Usage: shuf [OPTION]… [FILE]

Ways to specify a file

There is more than one way to specify the file using file shuf. Here are example commands for each way.

Implicit file as standard input

In this way we omit file from the arguments of the shuf command. Following the convention, you may opt to include a - in place of file to indicate the file to be taken as standard input.

Commands


seq 3 | shuf

Output

1
3
2

Explicit file as standard input

Commands


seq 3 | shuf -

Output

3
1
2

Notes

(1) Adding a - at the end of shuf serves as a visual cue improving readability in bash scripts.

File as the name of file

In this way, we specify a filename as file in the arguments of the shuf command. Here follow a few file shuf examples using files.

Shuffle input lines from the terminal

Commands


shuf /dev/fd/1

asdf
sdf
df
f
Ctrl-D

Output

df
f
asdf
sdf

Notes

(1) The above shuf command shuf /dev/fd/1 is equivalent to shuf -

(2) Termination of input lines through Ctrl-D is required

Shuffle lines in file

Commands


seq 3 > file;
shuf file;
rm -f file

Output

2
1
3

list shuf

In the last way to shuf, we operated on a file or input piped into the shuf command. In this way to shuf, we allow input lines to be specified as arguments of the shuf command using the -e option, forcing shuf to operate as list shuf.

Usage

Usage: shuf -e [OPTION]… [ARG]…

Ways to specify list args

Type input as args

Commands


shuf -e 1 2 3

Output

1
3
2

Notes
(1) The above shuf command  shuf -e 1 2 3 is equivalent to seq 3 | shuf -

Variable as args

Commands


var="1 2 3";
shuf -e $var

Output
[cc lang="bash"]
3
1
2

Parameter expansion as args

Commands


shuf -e 1… 3

Output

1
2
3

Command substitution as args

Commands


shuf -e $( seq 3 )

Output

3
2
1

range shuf

This last way to shuf is unlike the previous ways introduced. Instead of specifying a file or args in the command line, it requires a range of integers. The -i option, forces shuf to operate as range shuf.
Range shuf produces a range of integers in random order.

Usage

Usage: shuf -i LO-HI [OPTION]…

Ways to specify range

The one way: LO-HI

Commands


shuf -i 1-3

Output

2
3
1

Notes

(1) The shuf command shuf -i 1-3 is equivalent to all previous command using the sequence 1 2 3

Advanced shuf options

Here are some of the advanced options for shuf that may prove useful in bash programming.

Limit number of output lines

To limit the number of lines in the output, we use the -n option followed by an integer as follows.

Commands


shuf -i 1-3 -n 1

Output

3

Notes

Specify a file to write output lines

To specify a file to write output lines, we use the -o option followed by a filename as follows.

Commands


shuf -i 1-3 -n 1 -o file;
cat file;
rm -f file

Output

1

Notes

(1) The shuf command shuf -i 1-3 -n 1 -o file is equivalent to shuf -i 1-3 -n 1 > file using I/O redirection

Stream output lines

To create a continuous stream of output lines, we use the -r option as follows.

Commands


shuf -e 0,1 -r | xargs -i echo -n ""

Output

000101101010101101010110000101111010001010111001110…

Use the zero byte instead of newline as line delimiter

To use zero-terminated lines, we use the -z option as follows.

Commands


seq 3 | tr '\n' '\0' | shuf -z

Output

213

Notes

(1) The output contains non-printing zero byte between digits

How to shuf in bash the easy way

The easy way to shuf is to use the shuf command as discussed above. However, if you are a little curious about how you would shuf otherwise using other external commands or even pure bash, read on.

How to shuf the hard way

I have compiled a list of hard ways to shuf. Don't worry they are not that hard. They just don't make use of the shuf command.

Using sort

A common alternative to using file shuf is to use the sort command. Here's how:

Commands


seq 3 | sort -r

Output

2
3
1

Notes

(1) The shuf seq 3 | sort -r is equivalent to shuf -i 1-3

(2) shuf -i 1-3 is faster

Using gawk

Another alternative to using file shuf is to use the gawk command. Here's how:

Script

gawk-shuf()
gawk -v random=$RANDOM '
function randInt()
return int(rand()*1000)

function case_numeric_compare(i1, v1, i2, v2, l, r)
l = int(v1)
r = int(v2)
if(lelse if(l==r) return 0
else return 1

BEGIN
count=1
srand(random)


rank[count]=randInt()
line[count]=$(0)
count++

END
asorti(rank,order,"case_numeric_compare")
for(i=0;iprint line[order[i]]


' -

if [ $# -eq 0 ]
then
true
else
exit 1 # wrong args
fi
gawk-shuf

Source:  gawk-shuf.sh

Commands


seq 3 | bash gawk-shuf.sh

Output

2
3
1

Notes

(1) You may also use awk
(2) gawk-shuf.sh must exist in the working directory of commands

Pure bash

Script

pure-bash-shuf()
local line
local -a lines
while read -r line
do
lines[RANDOM]=$line
done
for line in $lines[@]
do
echo $line
done

if [ $# -eq 0 ]
then
true
else
exit 1 # wrong args
fi
pure-bash-shuf

Source: pure-bash-shuf.sh

Commands


seq 3 | bash pure-bash-shuf.sh

Output

2
3
1

Notes

  • The above script does not handle the case in which RANDOM happens to occur more than once. That is left as an exercise.

dJackblck17 The Game

I have put together a card game start called Jackblck17 based on blackjack for the purpose of showing the shuf command in action.

Script

deck()
echo A,2,3,4,5,6,7,8,9,10,J,Q,K-club,diamond,heart,spade

banner()
cat << EOF
bbbbbbbb
jjjj     b::::::b  lllllll  kkkkkkkk  1111111  77777777777777777777
j::::j   b::::::b  l:::::l  k::::::k  1::::::1 7::::::::::::::::::7
jjjj     b::::::b  l:::::l  k::::::k  1:::::::1 7::::::::::::::::::7
b:::::b  l:::::l   k::::::k 111:::::1  777777777777:::::::7
jjjjjjj  aaaaaaaaaaaaa  ccccccccccccccccb:::::bbbbbbbbb l::::l
cccccccccccccccc k:::::k    kkkkkkk   1::::1   7::::::7
j:::::j  a::::::::::::a   cc:::::::::::::::cb::::::::::::::bb
l::::l   cc:::::::::::::::c k:::::k   k:::::k    1::::1  7::::::7
j::::j  aaaaaaaaa:::::a c:::::::::::::::::cb::::::::::::::::b  l::::l
c:::::::::::::::::c k:::::k  k:::::k     1::::1  7::::::7
j::::j a::::ac:::::::cccccc:::::cb:::::bbbbb:::::::b l::::l
c:::::::cccccc:::::c k:::::k k:::::k   1::::l  7::::::7
j::::j    aaaaaaa:::::ac::::::c cccccccb:::::b    b::::::b l::::l c::::::c
ccccccc k::::::k:::::k       1::::l          7::::::7
j::::j  aa::::::::::::ac:::::c b:::::b   b:::::b l::::l c:::::c k:::::::::::k
1::::l         7::::::7
j::::j a::::aaaa::::::ac:::::c  b:::::b     b:::::b l::::l c:::::c
k:::::::::::k        1::::l        7::::::7
j::::ja::::a    a:::::ac::::::c   cccccccb:::::b   b:::::b l::::l c::::::c
ccccccc k::::::k:::::k       1::::l       7::::::7
j::::ja::::a    a:::::ac:::::::cccccc:::::cb:::::bbbbbb::::::bl::::::lc:::::::
cccccc:::::ck::::::k k:::::k   111::::::111   7::::::7
j::::ja:::::aaaa::::::a c:::::::::::::::::cb::::::::::::::::b l::::::l
c:::::::::::::::::ck::::::k  k:::::k  1::::::::::1  7::::::7
j::::j a::::::::::aa:::a cc:::::::::::::::cb:::::::::::::::b
l::::::l  cc:::::::::::::::ck::::::k   k:::::k 1::::::::::1 7::::::7
j::::j  aaaaaaaaaa  aaaa   ccccccccccccccccbbbbbbbbbbbbbbbb
llllllll  cccccccccccccccckkkkkkkk kkkkkkk11111111111177777777
j::::j
jjjj      j::::j
j::::jj   j:::::j
j::::::jjj::::::j
jj::::::::::::j
jjj::::::jjj
jjjjjj
EOF

score()
case $1 in
A) echo 0 ;;
[2-9]|10) echo $1 ;;
J|Q|K) echo 10 ;;
esac

score-hand()
local card
local points
local -i candidate_points
local -i aces
aces=0
for card in $hand
do
let points+=$( score $card/-*/ )
test ! "$card/-*/" = "A" ||
let aces+=1

done
test ! $aces -le 0 ||
echo $points

while [ $aces -gt 0 ]
do
for point in $points
do
new_point=""
for ace in 1 11
do
candidate_points=$(( point + ace ))
test ! $candidate_points -le 21 ||
echo "$candidate_points"
new_points="$new_points $candidate_points"

done
done
let aces-=1
points="$new_points"
done | sort -nur | head -1

jackblck17()
local deck
local card
local -i turn
local hand
deck=$( shuf -e $( deck ) )
banner
echo -e " Press enter key to continue"
read
turn=1
for card in $deck
do
test ! $turn -gt 2 ||
echo -e "\nhit or stay? (h) or s "
read
test ! "$REPLY" = "s" ||
break


echo -e "\n Your hand: \n"
hand="$hand $card"
echo " $hand"
test ! $( score-hand ) -gt 21 ||
echo -e "\nBust!\n"
exit

let turn++
sleep 1
done
echo -e "Dealer's hand: 17\n"
echo -e "Your hand: $( score-hand )\n"
test ! $( score-hand ) -gt 17 &&
echo -e "Dealer wins\n"
true
||
echo -e "You win!\n"


if [ $# -eq 0 ]
then
true
else
exit 1 # wrong args
fi
jackblck17

Source: jackblck17.sh

Commands

bash jackblck17.sh

Output

bbbbbbbb
jjjj   b::::::b   lllllll  kkkkkkkk 1111111   77777777777777777777
j::::j b::::::b   l:::::l  k::::::k 1::::::1  7::::::::::::::::::7
jjjj   b::::::b   l:::::l  k::::::k 1:::::::1 7::::::::::::::::::7
b:::::b   l:::::l  k::::::k 111:::::1 777777777777:::::::7
jjjjjjj aaaaaaaaaaaaa  ccccccccccccccccb:::::bbbbbbbbb     l::::l
cccccccccccccccc k:::::k    kkkkkkk   1::::1  7::::::7
j:::::j  a::::::::::::a  cc:::::::::::::::cb::::::::::::::bb l::::l
cc:::::::::::::::c k:::::k   k:::::k    1::::1 7::::::7
j::::j  aaaaaaaaa:::::a c:::::::::::::::::cb::::::::::::::::b  l::::l
c:::::::::::::::::c k:::::k  k:::::k     1::::1    7::::::7
j::::j           a::::ac:::::::cccccc:::::cb:::::bbbbb:::::::b l::::l
c:::::::cccccc:::::c k:::::k k:::::k      1::::l           7::::::7
j::::j    aaaaaaa:::::ac::::::c     cccccccb:::::b    b::::::b l::::l
c::::::c     ccccccc k::::::k:::::k       1::::l          7::::::7
j::::j  aa::::::::::::ac:::::c             b:::::b     b:::::b l::::l
c:::::c              k:::::::::::k        1::::l         7::::::7
j::::j a::::aaaa::::::ac:::::c             b:::::b     b:::::b l::::l
c:::::c              k:::::::::::k        1::::l        7::::::7
j::::ja::::a    a:::::ac::::::c     cccccccb:::::b     b:::::b l::::l
c::::::c     ccccccc k::::::k:::::k       1::::l       7::::::7
j::::ja::::a    a:::::ac:::::::cccccc:::::cb:::::bbbbbb::::::bl::::::lc
:::::::cccccc:::::ck::::::k k:::::k   111::::::111   7::::::7
j::::ja:::::aaaa::::::a c:::::::::::::::::cb::::::::::::::::b l::::::l
c:::::::::::::::::ck::::::k  k:::::k  1::::::::::1  7::::::7
j::::j a::::::::::aa:::a cc:::::::::::::::cb:::::::::::::::b  l::::::l
cc:::::::::::::::ck::::::k   k:::::k 1::::::::::1 7::::::7
j::::j  aaaaaaaaaa  aaaa   ccccccccccccccccbbbbbbbbbbbbbbbb
llllllll  cccccccccccccccckkkkkkkk  kkkkkkk11111111111177777777
j::::j
jjjj      j::::j
j::::jj   j:::::j
j::::::jjj::::::j
jj::::::::::::j
jjj::::::jjj
jjjjjj Press enter key to continue
Your hand:
3-heart
Your hand:
3-heart 4-spade
hit or stay? (h) or (s)
Your hand:
3-heart 4-spade 9-heart
hit or stay? (h) or s
s
Dealer's hand: 17
Your hand: 16
Dealer wins

The bottom line on shuf in bash

In this tutorial, we covered all you need to know about the shuf command and more. No longer will you have to result to do things the hard way, now that you have shuf.

To be honest, before writing this, I knew little that there was a command called shuf that could be used to randomize the order of any given input. After taking a deep dive into the shuf command for bash programming, Now, I can honestly say that it was worth it; shuf is more useful than I thought.

I hope that you enjoyed reading this as much as I enjoyed writing it and that it helps you in your career or homework. If it does, let me know.

Thanks,

Como instalar e usar o Veracrypt no Ubuntu 20.04
Se você não deseja que outras pessoas tenham acesso aos seus dados, a criptografia é essencial. Quando você criptografa seus dados confidenciais, pess...
Monitore o tráfego de rede com vnStat no Ubuntu 20.04
vnStat é um monitor de tráfego de rede leve e de código aberto para sistemas operacionais Linux. O vnStat monitora silenciosamente o tráfego de rede p...
Como criar e executar um script Perl no Ubuntu 20.04 LTS
Perl é abreviado para Practical Extraction and Reporting Language, que é uma linguagem bem conhecida e poderosa para processamento e manipulação de st...