27 August 2008 Veton Këpuska 2 Introduction Introduction to Digital State Machines Brief History Required Skills Data Structures and Algorithms 27 August 2008 Veton Këpuska 3 Course Outline ID: 682619
Download Presentation The PPT/PDF document "Digital State Machines Introduction" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Digital State Machines
IntroductionSlide2
27 August 2008
Veton Këpuska
2
Introduction
Introduction to Digital State Machines
Brief History
Required Skills – Data Structures and AlgorithmsSlide3
27 August 2008
Veton Këpuska
3
Course Outline
Introduction
Finite Automata & Regular Languages
Deterministic Finite State Machines
Non-Deterministic Finite State Machines
Regular Expressions and Languages
Context Free Grammars and Languages
Properties of Context-Free Languages
Introduction to Turing MachinesSlide4
Introduction to AutomataSlide5
Brief History – Early Days
Automata theory is the study of abstract computing devices.
Before advent of computers in 1930’s, Allan Turing studied an abstract machine with capabilities of today’s computers.
Goal to describe the boundaries of
What a computing device can do, and
What that computing device can not do.
Turing conclusions apply not only to his abstract machines but to today’s real computers.
27 August 2008
Veton Këpuska
5Slide6
Brief History – 1940’s & 1950’s
Simpler machines – “finite automata” were studied.
Models of brain functions were proposed
Useful for variety of other purposes
MIT’s Chomsky began the study of formal “grammars”
Formal grammars have a close relationships to abstract automata
They are basis of important parts of software including compilers.
27 August 2008
Veton Këpuska
6Slide7
27 August 2008
Veton Këpuska
7
Language, Thought and Turing Test
To many, the ability of computers to process language as skillfully as we humans do will signal the arrival of truly intelligent machines. The basis of this belief is the fact that the effective use of language is intertwined with our general cognitive abilities.
Among the first to consider the computational implications of this intimate connection was Alan Turing (1950). In this famous paper,
http://www.turing.org.uk/publications/testbook.html
http://www.abelard.org/turpap/turpap.php
:
Turing, A. (1950) “Computing Machinery and Intelligence”
Mind
59, 236, 433-60
Turing introduced what has come to be known as the
Turing Test
. Turing began with the thesis that the question of what it would mean for a machine to think was essentially unanswerable due to the inherent imprecision in the terms
machine
and
think
. Instead, he suggested an empirical test, a game, in which a computer’s use of language would form the basis for determining if it could think. If the machine could win the game it would be judged intelligent.Slide8
27 August 2008
Veton Këpuska
8
Language, Thought and Turing Test
In Turing’s game, there are three participants:
two people, and
a computer.
One of the people is a contestant and plays the role of an interrogator.
To win, the interrogator must determine which of the other two participants is the machine by asking a series of questions via a teletype. The task of the machine is to fool the interrogator into believing it is a person by responding as a person would to the interrogator’s questions. The task of the second human participant is to convince the interrogator that the other participant is the machine, and that they are not human.Slide9
27 August 2008
Veton Këpuska
9
Turing Test
The following interaction from Turing’s paper illustrates the kind of interactions he had in mind. Clearly a convincing simulation of humanity doesn’t require being an expert in all domains:
Q: Please write me a sonnet on the topic of the Forth Bridge.
A: Count me out on this one. I never could write poetry.
Q: Add 34957 to 70764.
A: (Pause about 30 seconds and then give answer as) 105621.Slide10
27 August 2008
Veton Këpuska
10
Turing Test
Turing predicted that roughly by the end of the twentieth century a machine with 10 gigabytes of memory would have around a 30%chance of fooling a human interrogator after 5 minutes of questioning. Given the fact that you can fool some of the people all the time, it is not clear how rigorous this particular standard is.
Nevertheless, the critical issue for Turing was that using language as humans do is sufficient, by itself, as an operational test for intelligence.Slide11
27 August 2008
Veton Këpuska
11
Turing Test
Surprisingly, many of the issues raised by Turing became relevant as early as 1966 in the context of the ELIZA program (named after Eliza Doolittle from George Bernard Shaw's
Pygmalion
; see
Weizenbaum, 1966
http://i5.nyu.edu/~mm64/x52.9265/january1966.html
).
ELIZA was an early natural language processing system capable of carrying on a limited form of conversation with a user.
Consider the following session with a version of ELIZA that imitated the responses of a Rogerian psychotherapist (virtual ideal female character).
User1: You are like my father in some ways.
ELIZA1: WHAT RESEMBLANCE DO YOU SEE
User2: You are not very aggressive but I think you don’t want me to
notice that.
ELIZA2: WHAT MAKES YOU THINK I AM NOT AGGRESSIVE
User3: You don’t argue with me.
ELIZA3: WHY DO YOU THINK I DON’T ARGUE WITH YOU
User4: You are afraid of me.
ELIZA4: DOES IT PLEASE YOU TO BELIEVE I AM AFRAID OF YOUSlide12
27 August 2008
Veton Këpuska
12
Turing Test
ELIZA is a remarkably simple program that makes use of pattern-matching to process the input and translate it into suitable outputs.
The success of this simple technique in this domain is due to the fact that ELIZA doesn’t actually need to
know
anything to mimic a Rogerian psychotherapist.
As Weizenbaum notes, this is one of the few dialogue genres where the listener can act as if they know nothing of the world.Slide13
27 August 2008
Veton Këpuska
13
Turing Test
Since 1991, an event known as the
Loebner Prize
competition has attempted to put various computer programs to the Turing test.
http://www.loebner.net/Prizef/loebner-prize.html
Although these contests seem to have little scientific interest, a consistent result over the years has been that even the crudest programs can fool some of the judges some of the time (Shieber,1994).
Not surprisingly, these results have done nothing to quell the ongoing debate over the suitability of the Turing test as a test for intelligence among philosophers and AI researchers (Searle, 1980).Slide14
27 August 2008
Veton Këpuska
14
Turing Test
Fortunately, for the purposes of this book, the relevance of these results does not hinge on whether or not computers will ever be intelligent, or understand natural language.
Far more important is recent related research in the social sciences that has confirmed another of Turing’s predictions from the same paper.
“Nevertheless I believe that at the end of the century the use of words and educated opinion will have altered so much that we will be able to speak of machines thinking without expecting to be contradicted.”Slide15
Turing Test
It is now clear that regardless of what people believe or know about the inner workings of computers, they talk about them and interact with them as social entities.
People act toward computers as if they were people; they are polite to them, treat them as team members, and expect among other things that computers should be able to understand their needs, and be capable of interacting with them naturally.Slide16
Brief History – 1969 and beyond
S. Cook’s study extended the findings of Turing - Separation of:
Problems that can be solved efficiently from those
Problems that can be solved in principle, but in practice take so much time that computers are useless for all but very small instances of the problems.
Intractable problems or “NP-hard”
Even with exponential improvement in computing speed it is unlikely that will enable us to solve large instances of intractable problems.
27 August 2008
Veton Këpuska
16Slide17
Brief History – 1969 and beyond
Thus theoretical development in this field allows us:
Understand if problems can be solved (i.e.,
coded
) efficiently, or
Focus on finding the way to approximate the solution, using heuristics, or some other method to limit the amount of time (intractability
) the program will spend solving the problem
Design and construct other important kinds of software.
27 August 2008
Veton Këpuska
17Slide18
Introduction to Finite AutomataSlide19
Introduction to Finite Automata
Finite Automata is comprised of a finite number of interconnected states.
The purpose of a state is to “remember” relevant portion of the system’s history.
Cons:
Finite number of states restrict the system to remember entire history.
Pros:Finite number of states can be implemented with fixed set of resources.
27 August 2008
Veton Këpuska
19Slide20
Example of Finite Automata
27 August 2008
Veton Këpuska
20
states
Final or Accepting state
Transition arcs labeled by inputs Slide21
Structural Representations
Grammars
Useful when designing software that processes data with a recursive structure.
Parsers – component of a compiler that deals with the recursively nested features of the typical programming language:
Expressions
ArithmeticControl Statements
They are expressed via rules:
E
E+E
Regular Expressions
Are used to denote structure of typically text data:
[A-Z][a-z]*[][A-Z][A-Z]
Describe Patterns
Newton MA
Cocoa Beach FL
[A-Z][a-z]*([] [A-Z][a-z]*)*[][A-Z][A-Z]
[A-Z] – represents range of characters from “A” to “Z”
[ ] – represents a blank (space) character
* - represents “any number of” preceding expression
27 August 2008
Veton Këpuska
21Slide22
Homework
Read the entire sections 1.2-1.4 on Formal Proof.
27 August 2008
Veton Këpuska
22Slide23
Central Concepts of Automata Theory
Alphabet
Strings
Languages
27 August 2008
Veton Këpuska
23Slide24
Alphabets
An alphabet,
, is a finite, nonempty set of symbols.
Common examples of alphabets:
= {0,1} – binary alphabet
= {a, b, c, …, z} – the set of lower-case (Latin) letters
The set of all ASCII characters, or the set of all printable ASCII characters.
27 August 2008
Veton Këpuska
24Slide25
Strings
A
string
(sometimes referred to as
word
), w
, is a finite sequence of symbols chosen from some alphabet.
01101011 – is a string from binary alphabet
= {0,1}.
The empty string,
∈
, is the string with zero occurrences of symbols.
Length of a string
w
is
|
w
|
:
|01101011| = 8
|
∈
| = 0
27 August 2008
Veton Këpuska
25Slide26
Strings
Powers of an Alphabet:
For a given alphabet
set of all strings of a certain length from that alphabet are denoted with an exponential notation.
k is defined to be a set of strings of length k, comprised from the set of symbols in .
27 August 2008
Veton Këpuska
26Slide27
Example
0
={
∈
}
, regardless of what alphabet is.
∈
is the only string whose length is 0.
If = {0,1} then
1
= {0,1}
2
= {00,01,10,11}
3
= {000,001,010,011, 100,101,110,111}
…
vs.
1
– alphabet
1
– set of strings of length 1.
27 August 2008
Veton Këpuska
27Slide28
Powers of an Alphabet
The set of all strings over an alphabet
is conventionally denoted by
*
For = {0,1}
*
= {0,1}
*
= {
∈, 0, 1, 00, 01, 10, 11, 000, …}, or
*
=
0
1
2
3
…
Excluding empty string from the set of all strings
*
of the alphabet is denoted by
+
:
+
=
1
2
3
…
*
=
0
+
=
{
∈
}
+
27 August 2008
Veton Këpuska
28Slide29
Concatenation of Strings
Let x and y be strings
xy
– denotes concatenation of x and y
If x = a1
a2a
3
…
a
i
,|x|=
i
&
y = b
1
b
2
b
3
…
b
j
,|y|=j
then
xy
= a
1
a
2
a
3
…a
i
b
1
b
2
b
3
…
b
j
,|
xy
|=
i+j
For
any string
w,
w = w = w
– identity for concatenation (analogous to identity for addition for numbers with 0)
Let x=01101 and y=110 then
xy
= 01101110, and
yx
= 11001101
27 August 2008
Veton Këpuska
29Slide30
Languages
A set of strings
L
all of which are chosen from
*, where
is a particular alphabet, is called a
language
.
If
is an alphabet, and
L
*
,
then
L
is a language over
Common languages (English, C/C++) are comprised of collection of words (English)/reserved words (C/C++) over the alphabet that consists of all the letters.
The alphabet of a language (English, C/C++) is a subset of the ASCII characters, thus they are subsets of the possible strings that can be formed from the alphabet of all ASCII characters.
27 August 2008
Veton Këpuska
30Slide31
Examples of Languages
The language of all strings consisting of n 0’s followed by n 1’s, for some n≥0:
{
,01,0011,000111,…}
The set of strings of 0’s and 1’s with an equal number of each
{
,01,10,0011,0101, 1001,…}
The set of binary numbers whose value is a prime
{
10,11,101, 111,1011,…}
*
is a language for any alphabet
, the empty language, is a language over any alphabet.
{}, the language consisting of only the empty string, is also a language over any alphabet.
Note: {}; the former has no strings and the latter has one (empty) string.
27 August 2008
Veton Këpuska
31Slide32
Automata Theory Problem
27 August 2008
Veton Këpuska
32Slide33
Problem
In automata theory, the
problem
is the question of deciding whether a given string is a member of some particular language.
Any problem in automata theory it turns out that it can be expressed as membership in a language.
Specifically, for an alphabet, and
L
a language over , the problem
L
is:
Given a string
w
in
*
, decide whether or not
w
is in
L.
27 August 2008
Veton Këpuska
33Slide34
Example
Determine if a number is a prime number.
This problem can be expressed by the language
L
p
consisting of all binary strings whose value as binary number is a prime.
27 August 2008
Veton Këpuska
34
Primary Number
?
1101
yes
no
Any solution to this problem will have to use significant computational resources of some kind: time and/or spaceSlide35
Note on the definition of a “Problem”
Commonly one thinks of a problem not as a decision question (is or is not the following true?) but as requests to compute or transform some input.
The task of a parser of a C/C++ compiler can be thought of as a problem in our formal sense:
Given an ASCII string parser has to decide whether or not the string is a member of L
c/c++
, that is the set of valid C/C++ programs.However, the compiler as a whole solves the problem of turning a C/C++ program into object code for some machine – which is significantly more than just answering “yes” or “no” about validity of a program.
Nevertheless, the definition of “problems” as languages has stood the test of time as the appropriate way to deal with the important question of complexity theory: Determining a lower bound on the complexity of certain problems.
27 August 2008
Veton Këpuska
35Slide36
End
27 August 2008
Veton Këpuska
36