Remaining Topics - PowerPoint Presentation

Slide1

Remaining Topics

Decidability Concept 4.1

The Halting Problem 4.2

P vs. NP 7.2 and 7.3

NP-completeness &

Cook-Levin Theorem 7.4Slide2

Review: Turing Machines in a nutshell

Church-Turing Thesis

Turing Machine

equal

Notion of an Algorithm

Turing Machine

Most simple machine possible

Computational power of

modern computer and

high-level language

Not particularly efficient in a practical senseSlide3

Review: Turing Machines in a nutshell

Most simply model possible…

Adding another tape can improve

efficiency (time or computational “speed”)

but not computational “power”

(ability to solve a problem).

Every multi-tape TM has an equivalent single-tape TM (

Theorem 3.13, p.149

)

Similar to an Automata

Non-determinism does NOT add any

“computational power” (

Theorem 3.15, p.150

)Slide4

Review: Turing Machines in a nutshell

Computational power of

modern computer and

high-level language

Every operation and statement in a high level language can be implemented with a

Turing Machine (TM)

Just as statements can be combined

So can TMs (HW5 illustrates this)Slide5

Decidability 4.1

Is there an algorithm that can decide if

An item is in a set.

A string is in a language

A formula is a member of a theory

These are all variations of the same concept, i.e., the concept of decidabilitySlide6

Decidability in Languages

We will concentrate on this:

Algorithms for deciding if a string is in a language

But, the strings and languages are going to represent deeper problems

A

DFA

= {<

B,w

> | B is a DFA that accepts input string

w

}Slide7

ADFA

A

DFA

=

{<

B,w

> | B is a DFA that accepts input string

w

}

B is the encoding of a DFA

Remember that you can encode a DFA as follows:

B = (Q,Σ,

δ

,

q

start

, F)

We are literally encoding the machine and the input (

w

) as a string

“<({1,2,3},{a,b},{(1,2,a),(1,3,b)},1,{3}),abc>”Slide8

ADFA

A

DFA

=

{<

B,w

> | B is a DFA that accepts input string

w

}

Testing whether a DFA accepts an input

w

is the same as the problem of testing whether the string <

B,w

> is a member of the language A

DFA

Just as A = {

w

|

w

= (11)*} would accept the set {

ε

, 11, 1111, 111111, …}

A

DFA

would enumerate all the <

B,w

>’

s

such that

w

is accepted by the encoded B.Slide9

ADFA

is decidable

A

DFA

=

{<

B,w

> | B is a DFA that accepts input string

w

}

What does this mean in plain English?

How can we prove it?Slide10

ADFA

is decidable

A

DFA

=

{<

B,w

> | B is a DFA that accepts input string

w

}

What does this mean in plain English?

“An algorithm exists that can accept strings that adhere to the definition of A

DFA

and reject string that don’t” Slide11

ADFA

is decidable

A

DFA

=

{<

B,w

> | B is a DFA that accepts input string

w

}

How can we prove it?

Proof is on p.167Slide12

ANFA

is decidable

A

NFA

=

{<

B,w

> | B is a NFA that accepts input string

w

}

How do we know this to be true?

Hint: How are

NFAs

and

DFAs

different?Slide13

AREX

is decidable

A

REX

=

{<

R,w

> | R is a Regular Expression that

generates

the string

w

}

How do we know this to be true?

Hint: How are Regular Expressions and

DFAs

related?Slide14

EDFA

is decidable

E

DFA

=

{<A> | A is a DFA and L(A) is empty}

Prove it

Hint: Just as Turing Machine can “simulate” a DFA it can also determine if a state is unreachable.Slide15

EQDFA

is decidable

EQ

DFA

=

{<A,B> | A and B are

DFAs

and L(A) = L(B)}

How do we know this to be true?

Hint: Symmetric Difference formulaSlide16

Decidability of Regular Languages

Deciding if

a language is

Regular

or not

If given DFA, NFA or REX

a

Regular

language is empty

two

Regular

languages are equalSlide17

Decidability of Context Free

Deciding if

a language is

Context Free

or not (Theorem 4.7)

If given CFG

a

Context Free

language is empty (Theorem 4.8)

two

Context Free

languages are equalSlide18

Classes of languages

Turing-recognized

Decidable

Context-Free (A

CFG

)

Regular (A

DFA

)Slide19

The Halting Problem

Will an algorithm halt on a given input.

Intuition:

Can you ever be sure that a loop is infinite?

It might just terminate in a few minutes, hours, years, millenniums, etc.

Sometimes you can make such a determination:

while (

x

> 0) {

x

=1;}

But is it always possible to make such a determination?Slide20

Infinite Looping

DFA:

by definition, upon consuming the input, the machine rejects unless it is in an accept state.

Looping is simply not an option by definition.

PDA:

very, very hard to make deterministic PDA’, but it can be done.

Once the input is consumed, empty transitions can move to a reject/accept state.

Every CF language has a PDA that will halt (not loop).

TM:

Just like a high-level language TMs can loop forever.

Intuition: you don’t consume the input, you can move on the tape infinitely, and the states can have a loop with no accept or reject.Slide21

ATM

A

TM

= {<

M,w

> | M is a TM and M accepts

w

}

U = “on input <

M,w

> simulate M on

w

”

If M accepts, U accepts

If M rejects, U rejects

Simple intuition:

M could be a Turing Machine that loops forever on certain input.

If M loops forever, U cannot be a decider for A

TMSlide22

Is A

TM

decidable?

A

TM

= {<

M,w

> | M is a TM and M accepts

w

}

U = “on input <

M,w

> simulate M on

w

”

If M accepts, U accepts

If M rejects, U rejects

BUT! Perhaps there is a way to implement M such that we can detect the infinite loop?

Upon infinite loop detection, U rejects.

U could still be a decider for A

TMSlide23

The Halting Problem

H(<

M,w

>) = if M accepts

w

accept

if M rejects

w

rejectSlide24

The Halting Problem is Undecidable

Proof: First, consider the machine/algorithm D:

D = “on input <M>, where M is a TM:

Run H on input <M,<M>>

Output the opposite of what H outputs; that is; if H accepts,

reject

and if H rejects,

accept.”

Recall H:

H(<

M,w

>) = if M accepts

w

accept

if M rejects

w

rejectSlide25

D is a crazy Decider Algorithm

D is implemented with a Turing Machine

D(<M>) = if D does not accept <M>, accept

if D accepts <M>, reject

What happens if we run D with its own Turing Machine description?

D(<D>) = if D does not accept <D>, accept

if D accepts <D>, rejectSlide26

A paradox emerges

D(<D>) = if D does not accept <D>, accept

if D accepts <D>, reject

If D accepts, how can D(<D>) reject?

We assumed that H could decide ATM because it could ‘somehow detect an infinite loop”

Think of H as a deterministic decider if a Turing Machine loops

Then, we use H to build D (the crazy decider)

Here we assume H can stop D from looping infinitely

Then, we run D on its own encoding, which creates a paradox.Slide27

Paradox resolved

Either H or D cannot exist.

Which one?

D is a TM machine that can simulate another Turing Machine, which has been elegantly proven.

Intuition

: Consider a program that can take another program and simulate its execution.

Program, Algorithm, and Turing Machine are all synonymous (Church-Turing Thesis)

Compilers

Virtual MachinesSlide28

Significance of Turing Machines

Turing Machines

are the “tool” we used to prove that the Halting Problem is un-decidable.

In other words, no algorithm exists to determine

if a

general

algorithm will halt or not.

Note: There are some algorithms where its easy to show/prove that it will halt, but we are interested in the

general

case (any/all algorithms).Slide29

Un-decidable Languages…

…there are many, but this is the interesting one:

A

TM

= {<

M,w

> | M is a TM and M accepts

w

}

Obviously, this language can’t be generated by a REX or CFG.

So, a NFA, DFA, and PDA can’t be used as a decider to accept/reject strings

But, even a

T

uring machine cannot act as a decider.

It may be able to decide some input on some machines, but not all.

There are strings in

A

TM

that will cause the decider to loop infinitely. Specifically <D,<D>> and likely other strings.Slide30

Significance of A

TM

A

formal language that cannot be decided by Turing Machine.

We can define this language’s concept

But we cannot create an algorithm (TM) to determine if a string is in this language or not.

A

TM

Turing-Decidable

Context-Free (A

CFG

)

Regular (A

DFA

)Slide31

Decidable vs. Recognizable

Turing Decidable Languages

Language such that some TM will

accept all

of its strings

And, reject strings in the language’s compliment

Halts on all input

Turing Recognizable Languages

Language such that some TM will

accept all

of its strings

But, might not halt on strings in the language’s compliment

Its it looping infinitely or will it accept?

We don’t know.Slide32

ATM

i

s Turing Recognizable

A

TM

= {<

M,w

> | M is a TM and M accepts

w

}

U = “on input <

M,w

> simulate M on

w

”

If M accepts, U accepts

If M rejects, U rejects

U will always halt if M halts. If M doesn’t halt on

w

than M doesn’t accept

w

, so <

M,w

> isn’t in the language. By its very definition U will always halt on strings in A

TM

.

The un-decidability is when U has been looping for 10 million years,

we really don’t know

Is it eventually going to be an accepted

w

or

an infinite loop caused by a rejected

w

. This is why infinity is trouble.Slide33

Time Complexity

TM’s are a formal way to describe algorithms

Some problems don’t have algorithms that will always halt, i.e., determining if a string is in ATM.

Algorithms that do halt can still take a long time.

How long is long?Slide34

General Time Unit

With Turing Machines we can define a unit of time to be the execution time of one TM transition.

With more practical machines, a time unit could be a CPU clock cycle, which might execute one machine-level instruction.

Some machines can execute 1 billion instructions per second, so the time unit would be 1/100000000 seconds.Slide35

Time as a function of input size

N is the size of the input

f(N

) is the number of time unit to solve the problem.

The running time of algorithms can be expressed as functions:

f(N

) = 2N + 5;

Two loops of size N and 5 setup instructions

Or, on loop of size N with two instruction inside and 5 instructions outside the loop.Slide36

Constants don’t matter

For really big problems, constants don’t matter

f(N

) = 2N is the same as

g(N

) = 100N

While 100 days seems like forever compared to 2 days, parallel computation and faster computers can eventually make up the difference (we hope).

For big problems,

f(N

) =

N

4

is much different than

g(N

) =

N

2

A faster computer may not help, why?Slide37

1.15 Days vs. 11 billion years

N

N^2

days

N^4

years

1

1

1.15741E-14

1

3.17E-17

2

4

4.62963E-14

16

5.07E-16

3

9

1.04167E-13

81

2.57E-15

4

16

1.85185E-13

256

8.12E-15

10

100

1.15741E-12

10000

3.17E-13

20

400

4.62963E-12

160000

5.07E-12

50

2500

2.89352E-11

6250000

1.98E-10

1000

1000000

1.15741E-08

1E+12

3.17E-05

10000

100000000

1.15741E-06

1E+16

3.17E-01

100000

10000000000

0.000115741

1E+20

3.17E+03

1000000

1E+12

0.011574074

1E+24

3.17E+07

10000000

1E+14

1.157407407

1E+28

3.17E+11Slide38

Big-O

Review

Constants don’t matter

Only the leading exponent matters

Why?Slide39

1.157407 days vs. 1.157419 days

N

N^2

days

N^2 + 100N

days

1

1

1.15741E-14

101

1.16898E-12

2

4

4.62963E-14

204

2.36111E-12

3

9

1.04167E-13

309

3.57639E-12

4

16

1.85185E-13

416

4.81481E-12

10

100

1.15741E-12

1100

1.27315E-11

20

400

4.62963E-12

2400

2.77778E-11

50

2500

2.89352E-11

7500

8.68056E-11

1000

1000000

1.15741E-08

1100000

1.27315E-08

10000

100000000

1.15741E-06

101000000

1.16898E-06

100000

10000000000

0.000115741

10010000000

0.000115856

1000000

1E+12

0.011574074

1.0001E+12

0.011575231

10000000

1E+14

1.157407407

1.00001E+14

1.157418981Slide40

Why do we only care about big N’s

Same reason I would worry about a $10,000 bill in my wallet but not a penny.

Same reason I would worry about a trip to Mars but not a trip to

Menands

.Slide41

Real Algorithm



TM Decider

Prepare for “hand-waving magic:”

Any algorithm that can be programmed can be reduced into a language problem.

A = {<

p,i,o

> |

p

is the encoding/description of a problem,

i

is the input, and

o

is the correct output.}

Deciding if a string is in L is the same thing as solving the problem.

The TM that decides A solves problem

p

.Slide42

The class P

The class of languages that can be decided in polynomial time.

Corresponds, the set of problems that can be solved in polynomial time.

Polynomial is

O(n

k

)

What are some problem in P that you have studied?Slide43

Did you know?

Every context free language is in PSlide44

The class NP

Non-deterministically Polynomial.

One way to think of this is NOT Polynomial.

Or, exponential

Or N!

But that is not the

whole story.