Tel Aviv University Started May 2015 Last Update February 5 2017 Comparators Can we use comparators to build efficient sorting networks Comparator networks ID: 759597
Download Presentation The PPT/PDF document "Sorting Networks Uri Zwick" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Sorting Networks
Uri Zwick
Tel Aviv University
Started:
May 2015
Last Update
:
February 5, 2017
Slide2Comparators
Can we use comparators to
build
efficient
sorting networks?
Slide3Comparator networks
A comparator network is a network composed of comparators.
There are input wires, each feeding a single comparator.
Each output of a comparator is either an output wire, or feeds a single comparator.
The network must be acyclic.
Number of output wires is also .
Slide4“Standard form”
Exercise:
Show that any comparator network
is equivalent to a network in standard form
with the same number of comparators.
For each compare/exchange,
the smaller item goes up,
the larger item goes down.
Slide5A simple sorting network
5 comparators
3 levels
Slide6Insertion sort
Slide7Selection/bubble sort
Slide8Selection/bubble Sort
Size =
Depth =
Slide9Exercise: Any sorting network that only compares adjacent lines must be of size at least .
Exercise:
Prove that the
odd-even transposition sort
, shown on the next slide,
is a sorting network.
Slide10Odd-Even Transposition Sort
Size =
Depth =
Slide11Theorem: If a network sort all 0-1 inputs,then it sort all inputs.
The 0-1
principle
Slide12The 0-1 principle
Lemma: Let be a monotone non-decreasing function. Then,if a network maps to ,then it maps to .
Proof: By induction on the number of comparisons using
Slide13The 0-1 principle
Suppose that a network is not a sorting network.
Proof:
It then maps some to , where , for some .
Let , iff , 0 otherwise.
The network maps to .
Thus, the network does
not
sort all
0-1
inputs.
Slide14Sorting by merging
Slide15Batcher’s odd-even merge
Slide16Batcher’s odd-even merge
even
Slide17Batcher’s odd-even merge
Slide18Batcher’s odd-even merge
To merge with :
Split into and
Split into and
Merge odd-indexed items to create
Merge even-indexed items to create
Compare/swap
Why does it work?
Slide19Batcher’s odd-even merge
Direct proof (without the 0-1 principle, distinct items)
Claim 1: should be in position or of merged sequence.
Claim 2: should be in position or of merged sequence.
Thus, the compare/swap of and puts all items in correct place.
Proof of Claim 1: Suppose comes from . (Other case similar.)
If , then Thus should be in position .
If , where , then , Thus, should is in position , if ,or in position , if .
The proof of Claim 2 is similar (exercise).
The -th odd item in
The -st odd item in
This item mayor may not exist
Slide20Batcher’s odd-even merge
Slide21Batcher’s odd-even merge -
Slide22Batcher’s odd-even merge -
Slide23Odd-even merge Odd-even sort
Slide24Odd-even merge for
Slide25Odd-even merge for
Slide26Batcher’s odd-even merge
Suppose that starts with 0’s and that starts with 0’s.
Then starts with 0’s,and starts 0’s.
The difference is either 0,1 or 2!
Proof using the 0-1 principle.
There is a problem only if difference is
2
and last level of comparators fixes it.
Slide27Batcher’s odd-even merge
0 0 0 0 0 1 1 1
0 0 0 0 0 1 1 1
0 0 0 0 1 1 1 1
0 0 0 0 0 1 1 1
0 0 0 1 1 1 1 1
0 0 0 0 0 1 1 1
Exercise:
Justify the use of the
0-1 principle for merging networks.
Slide28Batcher’s odd-even mergeSize – number of comparators
No better merging networks are known for any
!
Are the odd-even merge networks optimal?
,
Slide29Bitonic sequences
A sequence is bitonic iff it is acyclic shift of a strict bitonic sequence
8 6 4 1 2 5 7 9
4 1 2 5 7 9 8 6
A sequence is strict bitonic iff it is aconcatenation of a decreasing sequence and an increasing sequence
1 4 2 3
Slide30A Bitonic sorter
A (strict) bitonic sorter is a network that sorts every (strict) bitonic sequence.
I and are sorted,then is bitonic.
Thus, a strict bitonic sorter can serve as a merging network.
Slide31Batcher’s bitonic sorter
Is this different from odd-even merge?
Slide32A strict binary bitonic sequence –
The odd and even subsequences are also strict binary bitonic sequences.
Simple proof using the 0-1 principle.
The difference between the number of 0’s in the two sorted sequences is one of .
Batcher’s bitonic sorter
By induction they are sorted correctly.
Final level of comparators fixes the problem.
Slide33Very regular structure!
When
, there
are
levels with
comparators each.
Lines
and
are compared at level
iff
they differ
only
in the
-th most significant bit.
000
001
010
011
100
110
101
111
Batcher’s bitonic sorter for
=
Slide34Batcher’s bitonic sorter for =
Alternative recursive definition for
.
Sorts general
bitonic sequences.
(only when
.)
First level called
“half cleaner”
.
Slide35Batcher’s bitonic sorter for =
Alternative recursive definition for
.
Sorts general
bitonic sequences.
(only when
.)
First level called
“half cleaner”
.
Slide36Batcher’s bitonic merger for
=
Instead of
reversing
the first sequence, simply
adjust
the first level.
Slide37Batcher’s bitonic merge
Slide38Batcher’s merging and sorting networks
Odd-Even merge
Assume that
.
Smallest merging network known!
Bitonic
sort
Very regular structure.
Smaller depth for
.
Slide39Lower bound for merging networks
Let be the smallest possiblesize of an -merging network.
Top and bottom parts are both
-merging network.
Consider
There must be at least
comparisons between
top and bottom.
[Floyd (1969)]
Consider
[
Miltersen
-Paterson-
Tarui
(1996)]
Slide40Batcher’s Odd-even sort
Optimal size!
Slide41Batcher’s Odd-even sort
Size = 63
Optimal?
No!
Slide42Smallest known network for
[Green (1969)]
Knuth, Vol. 3, p. 227
Size =
60
Depth =
10
Slide43Smallest known sorting networks
Knuth (1998)
Vol. 3, p. 227
Slide448765432119161295310Batcher
8765432119161295310Batcher
For , the Batcher’s odd-even networks optimal.
For , Batcher’s networks are not optimal.
1615141312111096359534841373126Batcher6056514539352925Upper bound5349454137332925Lower bound
1615141312111096359534841373126Batcher6056514539352925Upper bound5349454137332925Lower bound
(See [Codish-(Cruz-Filipe)-(Schneider-Kamp) (2014)])
Smallest known sorting networks
Slide45Fastest
sorting networks
Slide46[
Codish
-(Cruz-Filipe)-Ehlers-Müller-(Schneider-Kamp) (2015)]
Optimal!
Slide47The AKS sorting networks[Ajtai-Komlós-Szemerédi (1983)]
There are sorting networks of depth ,and hence size .
Simplifications and improvements by[Paterson (1990)] [Pippenger (1990)][Chvátal (1991)] [Seiferas (2009)]
The constant factors are very large.
, [Chvátal (1991)]
Our presentation follows [Paterson (1990)] [Seiferas (2009)]
Focus on simplicity, not necessarily efficiency.
Slide48Sorting by splitting
Problem:
halving requires
depth.
Furthermore, prior to
AKS
, even
halvers
of
depth
were not known…
Slide49-Halver
Definition: For every input, and every ,at most of the smallest items end up at the bottom,and at most of the largest items end up at the top.
-
Slide50-Halver
Definition: For every input, and every ,at most of the smallest items end up at the bottom,and at most of the largest items end up at the top.
-
Slide51-Expander
A bipartite graph , where ,is an -expander if and only iffor every or with , we have .
Theorem: For every and ,there exists , such that for every there exists an -expander with vertices on each side.
Constant degree that does not depend on !
Slide52Probabilistic construction of expanders
Let , where , be obtained as the union of independently chosen random perfect matchings of and .
Theorem: If , then with positive probability is an -expander.
(The Entropy function)
Slide53Probabilistic construction of expanders
For
, , let be the probability that in .
,
where
,
.
The probability that
is
not
an
-
expander
is at most:
Slide54Probabilistic construction of expanders
,
Increasing in
.
Slide55-Expander (𝜀,𝛽)-Halver
An -expander , where ,composed of matchings, can be used to constructan (𝜀,𝛽)-halver of inputs of depth .
Each matching describes comparisons that can be performed in parallel in one round.
Slide56-Expander (𝜀,𝛽)-Halver
For a certain input and
, let
and
be the set of lines in which the largest items end up.
Lemma: The network does not contain a comparison between a line in and a line not in .
Items on lines of decrease with time.
Items on lines of increase with time.
Items in must be among the largest, at all times.
Items in must not be among the largest, at all times.
If a line in is compared with a line in , we get a contradiction.
Slide57Suppose that for a certain input, at least of the largest items, end up in .Let be the lines/registers in in which these items end up. .Let be the lines in containing the at most remaining largest items.Lines in could only have been compared with lines in .Thus, and ,so is not an -expander.
-Expander
(
𝜀,
𝛽
)
-
Halver
Slide58-Separator [Paterson (1990)]
For every
, at most of the smallest items are not in ,and at most of the largest items are not in .
-
Separator
For every
,
at most
of the
smallest
items are not in
,
and at
most
of the
largest
items
are not in
.
Left
fringe
Right
fringe
Slide59-Halvers -Separator
-
Halver
-
Halver
-
Halver
-
H
-
H
levels
Slide60The simplified AKS sorting network[Paterson (1990)]
Slide61The simplified AKS sorting network
[Paterson (1990
)]
Slide62Capacity
of a node at depth
at time
:
,
,
The simplified AKS sorting network
[Paterson (1990
)]
Some items are
sent back to the root.
Slide63The simplified AKS sorting network[Paterson (1990)] [Seiferas (2009)]
Capacity =
No. of items =
If
is odd, send an arbitrary item up.
At a leaf we should have
,
as items cannot be sent down.
Slide64The simplified AKS sorting network[Paterson (1990)] [Seiferas (2009)]
Three parameters (so far) determine the network: .
At the root, split item evenly and send to the two children.(Number of items in the root is always even.)
- Capacity of a node at depth at time .
Number of items in each node at each time is at most the capacity.
At a non-root node containing items:If , send all items up. Otherwise:If is odd, send an item up and let .Apply a separator with such that .Send the fringe items up.Send the central left/right items to left/right children.
Number of items is a power of two.
The network is modeled on a binary tree of depth
Slide65The depth of the tree is .
Depth and Time
The process goes on until the
capacity of a leaf
<
.
Number of time steps is
.
Slide66Let be such that .
Termination
Nodes at levels above
have capacity less than 1,
hence they are all
empty
.
At termination, capacity of leaves <
.
All items are in small subtrees of height
.
Each such subtree contains exactly
items.
IF
all items are in the correct subtree,
we can finish off by sorting each set of
items.
Slide67Strangers [AKS (1983)]
Number of items is a power of two.
Each node of the binary tree corresponds naturally to a subset of the items.(E.g., the left child of the root corresponds to the smallest items.)
An item currently in a node is nativeif it belongs to the set of items associated with the node.Otherwise, it is a stranger.
An item is a -stranger at a node if it has to move at least levels up the tree to become native.
If the items are , and a node at level is represented by an -bit string , then the subset of contains all items whose binary representation starts with .
If a -stranger, , is sent down, it becomes a -stranger.If a -stranger, , is sent up, it becomes a -stranger.If a native is sent to the correct child, it remains a native.If a native is sent to the wrong child, it becomes a -stranger.
Slide68Invariants and correctness [AKS (1983)] [Paterson (1990)] [Seiferas (2009)]
At even/odd times only even/odd levels are non-empty.Nodes at the same level contain the same number if items.Number of items in each node does not exceed the capacity.Number of -strangers in a node of capacity is at most .
At termination, the capacity of leaves, and hence all nodes < .
Number of strangers in each node , i.e., no strangers!
The network sorts, and has logarithmic depth!
Total Depth [Depth of )-separator]
Constant!
Slide69Capacity invariant
???
If
odd
If
, then all nodes above are empty and
must be even.
Otherwise,
.
Slide70Strangeness invariant ,
No. of
-
strangersat the children of
???
No. of
-
strangers
at the parent of
Fraction of these
that are sent to
Slide71Strangeness invariant ,
Added complication:
More than half of the items in
may belong to the set associated with
.
Thus, even if the split at
is perfect, some items
native
to
(and
) will be sent to
and become
-
strangers
.
How many
extra
items native to
can there be at
?
– the
parent
of
– the
sibling
of
Slide72Excess
If
more
than half of the
current
items in
are from
, this must be compensated by nodes
currently
containing
less
items of
than
in the “
ideal
” distribution.
Consider an “
ideal
” distribution of the items of
in the tree:
Items of
fill all the descendants of
(up to their current sizes), one-half
of
, one-eighth of
’s
grandparent
,
etc.
Compare the
current
distribution of items in
with the “ideal” distribution.
Let be the set of items native to .
Exercise:
Show that the “
ideal
” distribution does indeed exist.
Slide73Excess
Number of
items of
contained in the “
id
eal
” distribution in all ancestors of
is:
Items in descendants of
that do not belong to
are
strangers
.
Their number, therefore, is at most:
Let
be the set of items native to
.
Slide74Strangeness invariant ,
No. of
-
strangers
at the children of
No. of
-
strangers
at
,
“pushing” natives of
to
Excess no. of items in
that are native to
Clasiffication errors at
???
Note:
This term in
Seiferas
’ paper is
wrong
Slide75Required inequalities[Paterson (1990)] [Seiferas (2009)]
Total Depth [Depth of )-separator]
Sample parameters:
,
,
.With this choice .
Slide76Separators using -sorters [AKS (1992)]
Assume we have
-sorters, where
is a (large) constant.
We want to (approximately) split
items into
sets.
Sort each column
Only two levels
of
-sorters!
Split the rows in the desired proportions.
Randomly
permute rows
Sort each column
With appropriate parameters, the obtained network is an approximate separator, with positive probability.