Computability So far we talked about Turing Machines that decide languages and compute functions We only cared about making the machine decide the languagecompute the function We didnt care about the machines performance ID: 917663
Download Presentation The PPT/PDF document "Complexity & the O-Notation" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Complexity & the O-Notation
Slide2Computability
So far we talked about Turing Machines that decide languages and compute functions.
We only cared about making the machine decide the language-compute the function.
We didn’t care about the machine’s performance.
Slide3Time Complexity
The time complexity of a machine is the number of transitions it takes on input x in order to compute the function f(x) – accept or reject the input x.
The time is counted in terms of the length of the input. The time complexity function is a function t: N → N.
Slide4Worst and average cases
We say that a Turing machine has worst case time complexity t(n), if for every possible input x of length n the machine needs at most t(n) transitions in order to compute f(x)- decide if x is in the language.
The average case analysis is to consider all the running times on every possible input of length n and take the average.
Slide5Space Complexity
The space complexity of a Turing machine is very similar to the time complexity.
The idea is exactly the same. Instead of number of transitions we count the number of explored cells.
In the following examples the unexplored cells are shown in white and the explored ones are shown in blue.
Slide6Example: Put a $ before the input
One solution:
Repeat
Erase the first symbol from the input. Write it last in the output.
Until there is no symbol left in the input. Place a $.
1
0
1
1
Slide7Example: Put a $ before the input
One solution:
Repeat
Erase the first symbol from the input. Write it last in the output.
Until there is no symbol left in the input. Place a $.
1
0
1
1
Slide8Example: Put a $ before the input
One solution:
Repeat
Erase the first symbol from the input. Write it last in the output.
Until there is no symbol left in the input. Place a $.
0
1
1
1
Slide9Example: Put a $ before the input
One solution:
Repeat
Erase the first symbol from the input. Write it last in the output.
Until there is no symbol left in the input. Place a $.
1
1
1
0
Slide10Example: Put a $ before the input
One solution:
Repeat
Erase the first symbol from the input. Write it last in the output.
Until there is no symbol left in the input. Place a $.
1
1
0
1
Slide11Example: Put a $ before the input
One solution:
Repeat
Erase the first symbol from the input. Write it last in the output.
Until there is no symbol left in the input. Place a $.
$
1
1
0
1
Slide12Example: Put a $ before the input
The number of transitions you need for this solution is:
You repeat n times the following procedure (since n is the input length):
Erase the first symbol of the input
Move across the input and the output (about n transitions)
Paste this symbol in the last position of the output
Move across the output and the input (about n more transitions.
So the number of transitions in total is about 2n
2
Slide13Example: Put a $ before the input
The number of explored cells is exactly 2n+1
Slide14Example: Put a $ before the input
Another solution:
Erase the first symbol. Remember it. Replace it with a $.
Repeat:
Move one cell right. Replace this symbol with the one you remember from the left.
Until the input is consumed (you see a blank space). Put there the last symbol.
0
0
1
1
Slide15Example: Put a $ before the input
Another solution:
Erase the first symbol. Remember it. Replace it with a $.
Repeat:
Move one cell right. Replace this symbol with the one you remember from the left.
Until the input is consumed (you see a blank space). Put there the last symbol.
0
0
1
$
remember 1
Slide16Example: Put a $ before the input
Another solution:
Erase the first symbol. Remember it. Replace it with a $.
Repeat:
Move one cell right. Replace this symbol with the one you remember from the left.
Until the input is consumed (you see a blank space). Put there the last symbol.
1
0
1
$
remember 0
Slide17Example: Put a $ before the input
Another solution:
Erase the first symbol. Remember it. Replace it with a $.
Repeat:
Move one cell right. Replace this symbol with the one you remember from the left.
Until the input is consumed (you see a blank space). Put there the last symbol.
1
0
1
$
remember 0
Slide18Example: Put a $ before the input
Another solution:
Erase the first symbol. Remember it. Replace it with a $.
Repeat:
Move one cell right. Replace this symbol with the one you remember from the left.
Until the input is consumed (you see a blank space). Put there the last symbol.
1
0
0
$
remember 1
Slide19Example: Put a $ before the input
Another solution:
Erase the first symbol. Remember it. Replace it with a $.
Repeat:
Move one cell right. Replace this symbol with the one you remember from the left.
Until the input is consumed (you see a blank space). Put there the last symbol.
1
0
0
$
1
Slide20Example: Put a $ before the input
The number of transitions you need for this solution is:
Replace the first symbol with a $.
You repeat n times the following procedure (since n is the input length):
Replace the symbol we see with the one that was written in the left cell.
So the number of transitions in total is n+1
Slide21Example: Put a $ before the input
The number of explored cells is exactly n+1
Slide22Example: Put a $ before the input
The most efficient solution:
Just move left and place a $
0
0
1
1
Slide23Example: Put a $ before the input
The most efficient solution:
Just move left and place a $
0
0
1
1
$
Slide24Example: Put a $ before the input
The number of transitions is 2.
The number of explored cells is 2.
The time and space complexity of this machine doesn’t depend on the input (it is as we say “constant”). This is a very rare phenomenon! Most of the times we need at least to read all the input (so we need time and space at least n).
Slide25O-Notation
It is not always easy to count the exact complexity of a Turing Machine.
Furthermore sometimes we are just not interested in finding the exact number of transitions made or cells explored.
In those cases we perform as we say “asymptotic analysis”.
Slide26Asymptotic Analysis
In asymptotic analysis we completely ignore additive and multiplicative constants.
We also don’t care about small values. We want to see how the machine performs on large input.
Slide27O-Notation
We say that a function
f
is O(
g
(
n
))
(
or that
f
is upper bounded by
g) if there is a constant c>0 and an integer n
0 such that:
n
0
c
∙
g
f
Slide28O-Notation
We say that a function
f
is
Ω
(
g
(
n
))
(
or that
f is lower bounded by g) if there is a constant c
>0 and an integer n0 such that:
n
0
c
∙
g
f
Slide29O-Notation
We say that a function
f
is
Θ
(
g
(
n
))
(
or that
f is upper and lower bounded by g) if there are constants c
1 ,c2 >0 and an integer n0 such that:
n
0
c
1
∙
g
f
c
2
∙
g
Slide30O-Notation - Properties
If
f
= O(
g
) then
g
=
Ω
(
f
)
Since f(n
) = O(g(n)) there is a constant c and an integer n0
such that for all n ≥ n0 , f(n
) ≤ c ∙ g(n)So, that means that for all
n ≥ n0 , g(n
) ≥ 1/c ∙ f(n).Thus
g(n) = Ω(f(
n))If f = O(g
) and
f
=
Ω(
g
) then
f
=
Θ
(
g
).
Since
f
(
n
) =
Ω
(
g
(
n
)) there is a constant
c
1
and an integer
n
1
such that for all
n
≥
n
1
,
f
(
n
) ≥
c
1
∙
g
(
n
)
Since
f
(
n
) = O(
g
(
n
)) there is a constant
c
2
and an integer
n
2
such that for all
n
≥
n
2
,
f
(
n
) ≤
c
2
∙
g
(
n
)
Take
n
0
= max{
n
1
,
n
2
}. Then for all
n
≥
n
0
,
c
1
∙
g
(
n
) ≤
f
(
n
) ≤
c
2
∙
g
(
n
). So
f
(
n
) =
Θ(
g
(
n
))
Slide31The o- and ω-
symbols
We say that a function
f
is o(
g
(
n
)) if for every constant
c
>0 there is an integer
n
0 such that:We say that a function
f is ω(g(n)) if for every constant c
>0 there is an integer n0 such that:
Slide32The o- and ω-
symbols
Another way to prove that f is o(g) or
ω
(g) is by using limits.
A function
f
is o(
g
(
n
)) if:
A function f
is ω(g(n)) if:
Slide33The o- and ω-
symbols
o and
ω
have the same relation with O and
Ω
as < and > have with ≤ and ≥.
Θ
stands for ≈
If f = O(g) but f ≠
Θ(
g) then we say that f = o(g)
Similarly, if f = Ω(
g) but f ≠ Θ(g) then f = ω(g)It is important to understand that if a function f is o(g) this doesn’t mean that f is going to be less than g for every input but that there is some input after which
f is always less than g.
Slide34Examples
10n = O(n
2
) because for c=1 and n
0
= 10, for all n≥n
0
10n ≤ n
2
.
Furthermore 10n is o(n
2
) because for all c>0 there is a n0
(n0 = 10/c) such that for all n≥n0 , 10n ≤ cn2.
An other way to see this is by taking the limitThe function n2 is considered greater than 10n besides the fact that for some small inputs (like n=2) 10n > n.
Slide35Examples
1000n
2
= o(2
n
). That is because:
Slide36Examples
log
2
n =
Θ(
log
10
n).
There is a property of the logarithms:
log
x
n = log
yn ∙ logxy. So log
2n = log210 ∙ log10n But log2
10 is a constant, 3 ≤ log210 ≤ 4. So for all n≥1, 3∙log10n ≤ log2n ≤ 4∙log
10nThe logarithms are a constant factor apart so whenever we right log n as a complexity function we generally ignore the base.
Slide37O- relation of several functions
The polynomial
a
k
n
k
+ a
k-1
n
k-1
+ … + a
1
n + a0 is
Θ(nk) (the highest power conquers).If c1, c
2 are constants, 1 < c1 < c2 then nc1 =o(nc2
)If c1, c2 are constants, c1 < c
2 < 1 then nc2 =o(nc1)If c1
, c2 are constants, 1 < c1 < c2 then c
1n =o(c2n) For any constants c>1 and c‘,
nc’ = o(cn
)
For any constants 1 < c
1
< c
2
, log
c1
n =
Θ
(log
c2
n)
Slide38Complexity Classes
The class DTIME(t(n)) contains all those languages L for which there is a DTM that decides L in time O(t(n)) (i.e. performing O(t(n)) steps)
The class DSPACE(t(n)) contains all those languages L for which there is a DTM that decides L exploring O(t(n)) cells in total