Holland 1986 Classifier systems rulebased systems with general mechanisms to process rules in parallel for the adaptive generation of rules and for testing the effectiveness of existing rules ID: 529630
Download Presentation The PPT/PDF document "Bull-Paper Review1" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Bull-Paper Review1
Holland (1986): “
Classifier systems … rule-based systems with general mechanisms to process rules in parallel, for the adaptive generation of rules, and for testing the effectiveness of existing rules.”
LCS
= Reinforcement Learning
Evolutionary Computing heuristics to produce adaptive systems
Bull on EC: “
the population of candidate solutions is seen to adapt to the problem
”
Reinforcement learning
attempts to map state action combinations to their utility, with the aim to maximize future rewards.
EC is used to search the space of possible rules, while reinforcement learning techniques are used to assign utilities to existing rules, thereby guiding the search for better rules.Slide2
Bull-Paper Review2
Holland’s LCS: bids of successfully rules are placed in
a
bucket and reinforcement learning redistributes these bids between subsequent chosen rules. As an example how this redistribution is done for ZCS read pages 20-21 of the textbook.
For understanding ZCS read the textbook and not Bull’s paper who fails to explain its mechanisms clearly.
Bull on the difference between ZCS and XCS: The most significant difference between XCS and other systems it its
intention to form an accurate mapping of the problem space
.
ZCS/Holland employ TD(0) whereas XCS employs Q-learning
to be explained next week when
The second paragraph of the textbook on page 124 assumes a more complex model of XCS in which a state space will be explored
to be explained next week; as of now, just assume that the payoff of an action is the reward it receives: P=RSlide3
Bull-Paper Review3
Bull’s characterization how deletion works in XCS is misleading in that is suggest as is the only thing used
—most implementation use as/F, as suggested earlier
Different variations of LCS differ in to which sets of rules operations are applied:
[N]: all rules (also sometimes called [P])
[M]: rules that match the input message
[A]: rule(s) that are selected to process the incoming message
Application of LCS-style systems (mostly in optimization, control, and modeling
—I am not convinced they are too many applications in data mining except “
mining data streams
”):
Stock price forecasting
Fighter aircraft modeling
Control of routing nodes in package switch networks
Control in electrical power distribution
Traffic Signal Control
XCS Software (there likely are
newer versions!):
http://www2.cs.uh.edu/~
ceick/6367/Butz-XCS.pdf