Internet Engineering
Jacek Mazurkiewicz, PhD
Softcomputing
Formal Introduction
•
contact hours, room No. 225 building C-3:
Monday: 12:45 - 15:15, Friday: 14:30 - 16:00,
•
slides:
www.zsk.ict.pwr.wroc.pl
•
„Professor Wiktor Zin”
•
test: 25.01.2016 during lecture
- softcomputing:
- lecture + laboratory
- laboratory mark – 20% of final mark
- bonus question!
Program
• Idea of intelligent processing• Fuzzy sets and approximate reasoning
• Expert systems - knowledge base organization
• Expert systems - reasoning rules creation
• Expert systems: typical organization and applications
• Artificial neural networks: learning and retrieving algorithms
• Multilayer percetpron
• Kohonen neural network
• Hopfield neural network
• Hamming neural network
• Artificial neural networks: applications
• Genetic algorithms: description and classification
SUBJECT OBJECTIVES C1. Knowledge of artificial neural networks in pattern recognition, digital signals
and data processing: topology of networks, influence of parameters for network behavior. C2. Knowledge of genetic algorithms used for data pre- and postprocessing.
C3. Knowledge of expert systems – reasoning rules and knowledge base creation for different tasks. C4. Skills of special environment usage for project phase, modeling and simulation
of softcomputing systems in case of different scientific problems.
SUBJECT EDUCATIONAL EFFECTS
relating to knowledge:
PEK_W01 – knows the rules and the idea of intelligent processing.
PEK_W02 – defines the fuzzy sets and understands the idea of approximate reasoning.
PEK_W03 – defines the knowledge base and reasoning rules, knows the expert systems construction.
PEK_W04 – knows the architecture of typical artificial neural networks structures, learning and retrieving algorithms, applications.
PEK_W05 – knows the description, classification, examples of applications of genetic algorithms
relating to skills:
PEK_U01 – can use the environments for project phase, modeling and simulation of artificial neural networks as well as genetic algorithms in different tasks about pattern digital signals recognition.
PEK_U02 – can use the environments for project phase, modeling and implementation of expert systems to dedicated fields of knowledge.
PEK_U03 – can use the environments for project phase, modeling and implementation of fuzzy sets and fuzzy reasoning to dedicated fields of knowledge.
Literature
• B. Bouchon Meunier, Fuzzy Logic and Soft Computing
• O. Castilo, A. Bonarini, Soft Computing Applications
• M. Caudill, Ch. Butler, Understanding Neural Networks
• E. Damiani, Soft Computing in Software Engineering
• R. Hecht-Nielsen, Neurocomputing
• S. Y. Kung, Digital Neural Networks
• D. K. Pratihar, Soft Computing
• S. N. Sivanandam, S. N. Deepa, Principles of Soft Computing
• A. K. Srivastava, Soft Computing
• D. A. Waterman, A Guide to Expert Systems
Why Neural Networks and Company?
Still in active use
No chance to solve some problems in other way Human ability vs. classical programs
Works as primitive human’s brain Artificial intelligence has power!
ANN + Fuzzy Logic + Expert Systems + Rough Sets + Ant Algorithms = SoftComputing
The Story
1943 – McCulloch & Pitts
– model of artificial neuron
1949 – Hebb
– information stored by biological neural nets
1958 – Rosenblatt
– perceptron model
1960 – Widrow & Hoff
– first neurocomputer - Madaline
1969 – Minsky & Papert
– XOR problem – single-layer perceptron limitations
1986 – McCleland & Rumelhart
Where Softcomputing is in Use?
Letters, signs, characters, digits recognition Recognition of ship types – data from sonar Electric power prediction
Different kinds of simulators and computer games Engine diagnostic – in planes, vehicles
Rock-type identification Bomb searching devices
Neural Networks Realisation
Set of connected identical neurons
Artificial neuron based on a biological neuron Hardware realisation – digital device
Software realisation – simulators
Artificial neural network – idea, algorithm, mathematical formulas Works in parallel
Learning
With a Teacher Without a Teacher Klasyfikator Wektor cech (dane nauki) Wynik klasyfikacji NauczycielTeacher Learning vector Parameters Weights Result of learning Klasyfikator Wektor cech (dane testowe) Wynik klasyfikacji Learning vector Result of learning Parameters WeightsSoftcomputing vs. Classical Computer
Different limitations of softcomputing methods No softcomputing:
– operations based on symbols: editors, algebraic equations – calculations with a high level of precision
Anatomy Foundations (1)
Nervous System – 2-ways, symmetrical set of structures, divided into 4 parts:
Spinal Cord
– receiving and transmission of data
Prolonged Cord
– breathing, blood system, digestion
Cerebellum
– movement control
Brain (ca. 1.3 kg) – 2 hemispheres
– feeling, thinking, movement
brain
brain stern cerebellum prolonged cord
spinal cord
Anatomy Foundations (3)
Cerebral cortex – thickness: 2 mm, area: ca. 1.5 m2
Cerebral cortex divided into 4 part – lobes Each lobe is corrugated
Each hemisphere is responsible for half part of body: right for left part, left for right part
Hemispheres are identical in case of a structure, but their functions are different
Anatomy Foundations (4)
Brain composed by fibres with large number of branches Two types of cells in nervous tissue: neurons and gley cells There are more gley cells:
– no data transfer among neurons – catering functions
Ca. 20 milliard neurons in cerebral cortex Ca. 100 milliard neurons in whole brain
Neuron: dendrites – inputs, axon – output, body of neuron
Anatomy Foundations (5)
Neurons in work:
• chemical-electrical signal transferring • cell generates electrical signals
• electric pulse is changed into a chemical signal at the end of axon • chemical info passed by neurotransmitters
• 50 different types of neurons
• neurons driven by a frequency of hundreds of Hz • neurons are rather low devices!
Biological and Artificial Neural Nets
Artificial neural networks are a good solution for:
– testing already identified biological systems
– pattern recognition
– alternative configurations to find the basic features of them Artificial neural networks are primitive brothers of biological nets
Biological nets have sophisticated internal features important for their normal work Biological nets have sophisticated time dependences ignored in most artificial networks Biological connections among neurons are different and complicated
Most architectures of artificial nets are unrealistic from the biology point of view Most learning rules for artificial networks are unreal in biology point of view
Most biological nets we can compare to already learned artificial nets to realise function described in a very detailed way
Linear ANN - ADALINE (ADAive Linear Neuron)
single neuron’s answer:
+
. . . x x x 1 2 M w w w 1 2 M w0 y 1
M j j jx w w y 1 0M – number of input neurons
K – number of output neurons
M j j jx w y 0 ~ ) ~ (x wTx ) ,..., , ( ~ 1 0 x xM x col x 1 0 x ) ,..., , (w0 w1 wM col wscalar description vector description
Single-Layer Multi-Output Network
1 x x x1 2 M y y1 2 Ky w10 w20 wK0 w w w11 12 1K w w w21 22 2K wM1 wM2 wMKW
kj Output neuron Input neuron k-neuron’s answer:
M j j kj K w x y 0 ) (x column w x y(x) T y(X) WX KM K K M M w w w w w w w w w 1 0 2 21 20 1 11 10 WLearning Procedure
experimental data: N - series
N x x x1, 2,..., N K K K t t t1 , 2 ,..., – learning data – required answers , N K N t
x – function implemented by net
error function – mean-square error:
N n K k n k k t y W E 1 1 2 2 1 ) ( w
N n K k M j n k n j jk x t w W E 1 1 2 0 2 1 ) (looking for a minimum of E(W) function:
0 ) ( ,