103_6_full - PSYC 103 Winter 2011 Lecture 6 Review Session...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: PSYC 103 Winter 2011 Lecture 6 Review Session Pepper Canyon Hall room 109 from 8-9:20pm on January 18th Establishing an Operant Response: Shaping Shaping: the gradual change from some initial behavior to some desired target response; reinforcement of successive approximations Shaping new responses - variability is key Operant (Instrumental) Conditioning In pavlovian conditioning: US UR, CS US UR, CS UR In Instrumental Conditioning, presentation of US depends on the animal’s behavior The response is ‘Instrumental’ in producing the outcome. Stimulus → Response → Outcome Instrumental or Operant Behavior reinforcer The Conditions of Learning R-O contingencies Contingency Positive: Response receives reinforcer Reinforcing consequence Pleasant Aversive Positive reinforcement Punishment Response increases Response decreases Negative: Response prevents reinforcer Omission training: response cost Response decreases Negative reinforcement Escape Response increases The Conditions of Learning R-O contingencies Stimulus → Response → Outcome Positive Reinforcement: Delivery of a stimulus shortly following a response to increase the future probability of that response. A hungry rat presses a bar to receives food. The rat's behavior of pressing the bar is strengthened by the consequence of receiving food. Negative Reinforcement: Removal of an aversive stimulus shortly following a response to increase the future probability of that response. A rat presses a bar to stop a foot shock. Punishment: Delivery of an aversive stimulus shortly following a response to decrease the future probability of that response. A rat presses a bar and receives a mild electrical shock on its feet. The consequences of behavior Reinforcer Stimulus → Response → Outcome any event that strengthens the behavior it follows Primary Reinforcer (typically) Pleasing stimulus Motivation: satisfies a biological need (food, water, sex, drugs) Size and Temporal proximity matter: Immediate reinforcement is most effective Can lead to superstition Secondary Reinforcers: Conditioned reinforcer: neutral stimulus paired with primary reinforcer Overcomes ineffectiveness of delayed reinforcement Marking stimulus (token reinforcer): neutral stimulus given in lieu of Rf. Can also overcome effects of delayed reinforcement The Conditions of Learning Contiguity: temporal relationship between response & reinforcer Close contiguity of the response & reinforcer is best … -  Learning occurs when response is immediately followed by reinforcement -  Logan (1960) Trained rats to run down an alley for food: Speed of running was fastest if food available immediately rat enters goal box. Speed of running directly related to delay: Gradient of delay. .. But not always necessary Instrumental responding still possible with a considerable delay: Delay of 30 sec between response and reinforcer Rate of responding was slow, but was still maintained. 8 Conditions of learning Contingency: probabilistic relationship between response & reinforcer Strong contingency leads to higher rates of responding Hammond (1980) -  Experimental session divided into 1-sec intervals -  If rat presses lever, US delivered at the end of that 1-sec interval, 12% of the time At end of training, response rate ≈ 50 responses/min -  Another group given identical training, but US also delivered during intervals that did not contain a response, still 12% of the time At end of training, response rate ≈ 15 responses/min Two reasons why: 1. Molar theory of reinforcement Instrumental conditioning effective IF responding increases the overall rate of reinforcement BUT, Thomas (1981) showed instrumental conditioning effective in rats when: -  lever pressing had no effect on rate of reinforcement -  lever pressing decreased the rate of reinforcement 2. Molecular theory of reinforcement Instrumental conditioning effective IF a response is specifically tied to the reinforcer 9 Conditions of learning What makes a good reinforcer? (1) Thorndike: A satisfying state of affairs: “One which the animal does nothing to avoid, often doing things which maintain or renew it “ Circular definition! (2) Premack Principle Premack (1959) Reinforcers = opportunities to engage in behavior, e.g. eating If two activities (A and B) are freely available, and A is chosen over B, then A is said to be a reinforcer for B (3) Can only be determined empirically (skinner) 10 The nature of instrumental learning: what is learned Thorndike: S R association -  Law of effect: Whenever a response is followed by a reinforcer it strengthens an S-R connection - Formed basis of theorizing by Guthrie (1935) and Hull (1943) -  Unintuitive? Does not allow animals to anticipate outcome Tolman (1932) & Elliot (1928) : R US association -  Animals form R-US associations, they “expect” a certain outcome - Rats trained to run down an alley for food - Changing the value of the reinforcer influenced the level of responding - Can use reinforcer (US) devaluation to show this (see examples in book) 11 What does the stimulus do? Stimulus → Response → Outcome S-R association: Law of Effect Acts as catalyst S-O association: Clark Hull 1930’s Similar to CS-US association in Pavlovian cond. S (R - O) Three-Term Contingency: Skinner SD signals the existence of the R-O contingency SD is a discriminative stimulus that “sets the occasion” for R-O There is experimental evidence for all four types of associations: R-O, S-R, S-O, S(R-O) Constraints on learning •  Strong Behaviorist Prediction: if reinforcement is given, any response can be conditioned •  Thorndike: Law of Effect •  J.B. Watson •  Experimental Results: Some behaviors are more easily conditioned than others –  Very hard for cats to learn to yawn or scratch to get out the puzzle box. –  coin (or token) release is hard to train (Breland and Breland) •  S - O association dominates for in this case Schedules of reinforcement When P(O|R) = 1, every occurrence of the response is reinforced What if P(O|R) ≠1? Possible scenarios •  Response must be repeated before reinforcement is obtained •  Reinforcement is given only after a certain amount of time has passed Schedules of reinforcement: rules that defines which occurrences of the instrumental response are reinforced; primary determinants of behavior • Ratio Schedule: The number of responses determines reinforcement • Interval Schedule: The timing of the response (since the last reinforcer) determines reinforcement Cumulative record No responding High response rate No responding Low response rate Slope of the cumulative record is directly proportional to the rate of responding Ratio Schedules The number of responses determines reinforcement, the time between response does not matter Fixed ratio (FR): required number of responses is constant Variable ratio (VR): required number of responses varies between reinforcer deliveries Interval Schedules The passage of time determines the availability of reinforcement; the number of responses made during the interval does not matter. Fixed Interval (FI): required interval is constant Variable Interval (VI): required interval varies between reinforcer deliveries FI scallop Typically, once the interval expires the animal must make one response Interval schedules produce lower rates of responding than ratio schedules (iv) Law of effect and problem solving Insight Kohler (1925) -  Hung a banana from the ceiling of a cage, out of reach of 6 apes. Wooden box in cage. - Period of inactivity, then one chimpanzee suddenly moved a box towards the banana and then climbed on the box to reach the banana - Was this problem solved by trial and error? -  Difficult to say, apes had prior experience with boxes, sticks, so… may have learned through trial and error, may not. -  Epstein et al. (1984): Highlights the importance of prior experience on seemingly insightful behavior 19 (iv) Law of effect and problem solving Causal inference and folk physics Do animals have an appreciation of the physical and causal properties of the objects they are using? Primates Premack (1976) -  Chimpanzee had to replace the shape in the middle of the upper row with the knife to gain reward -  Reveals the animal understood that the knife cuts apples… -  BUT could be explained by the animal’s past experience of seeing knives and apples together -  FURTHERMORE Povinelli (2000) cites 27 experiments, all showing chimps have no understanding of physical properties of the problems they were solving 20 Causal inference and folk physics Diagram of the apparatus used by Povinelli (2000) and by Visalberghi and Limongelli (1994) to test whether an animal will push a peanut in the direction that ensures it does not fall into a trap. From Visalberghi and Limongelli, 1994. Copyright © 1994 American Psychological Association. Reproduced with permission. Causal inference and folk physics Diagram of the apparatus used by Heinrich and Bugnyar (2005). A raven stood on the perch and was expected to retrieve food by pulling the string upwards (left-hand side) or downwards (right-hand side). ...
View Full Document

This note was uploaded on 03/24/2012 for the course PSYC 103 taught by Professor Pearlberg during the Spring '07 term at UCSD.

Ask a homework question - tutors are online