Behavioral contrast when responses are maintained by unsignaled delayed reinforcement

 

Contraste conductual cuando las respuestas son mantenidas por el reforzamiento demorado no señalado

 

Kennon A. Lattal and Julie M. Smith

 

Department of Psychology, West Virginia University.

 

Corresponding author:
Kennon A. Lattal,
Department of Psychology, West Virginia University,
Morgantown, West Virginia, 26506-6040, USA.
E-mail: klattal@wvu.edu

 

Received: December 24, 2011
Final acceptance: December 30, 2011

 

Abstract

Positive and negative behavioral contrast were studied in pigeons when keypecking was maintained by a variable-interval (VI) schedule with a 1 - or 2-s unsignaled delay prior to each reinforcer. In three different conditions, extinction (defined as removing the reinforcer) alternated with the VI schedule. These extinction conditions were programmed as 20-s blackouts of the chamber, 100-s blackouts of the chamber, or 20-s changes in the color of the response key. Adding extinction in any of these forms increased response rates maintained by the delayed-reinforcement contingency (positive contrast), and removing extinction decreased these response rates (negative contrast). Differential behavioral contrast did not occur systematically across pigeons as a function of the different extinction conditions. The results extend to responding maintained by delayed reinforcement the range of circumstances under which behavioral contrast occurs. The role of contingencies controlling low response rates and different types and durations of stimuli in generating behavioral contrast are discussed.

Keywords: positive behavioral contrast, negative behavioral contrast, unsignaled delay of reinforcement, signaled delay of reinforcement, extinction, blackout duration, key peck, pigeons.

 

Resumen

Se estudió el contraste conductual positivo y negativo en palomas, cuando las respuestas a una tecla se mantuvieron empleando un programa de reforzamiento de intervalo variable (IV), con una demora de 1 ó 2 s antes de la presentación de cada reforzador. En tres diferentes condiciones, un período de extinción (definido como la suspensión del reforzador) alternó con el programa de IV. Estas condiciones de extinción se programaron como blackouts de 20 s en la cámara experimental, blackouts de 100 s en la cámara experimental o como un cambio en el color de la tecla de respuesta durante 20 s. Añadir el período de extinción en cualquiera de éstas formas resultó en aumentos en las tasas de respuesta mantenidas por la contingencia de reforzamiento demorado (contraste positivo) y eliminar los períodos de extinción resultó en disminuciones de las tasas de respuesta (contraste negativo). El contraste conductual diferencial no ocurrió sistemáticamente a través de las palomas en función de las diferentes condiciones de extinción. Los resultados extienden la variedad de las circunstancias bajo las cuales el ocurre el contraste conductual al caso de las respuestas mantenidas por el reforzamiento demorado. Se discute el papel de las contingencias que controlan tasas bajas de respuesta y de los diferentes tipos y duraciones de los estímulos para generar contraste conductual.

Palabras clave: contraste conductual positivo, contraste conductual negativo, demora de reforzamiento no señalada, demora de reforzamiento señalada, extinción, duración del blackout, presiones a la tecla, palomas.

 

The focus of investigations of behavioral contrast has been on the effects of manipulating variables in the changing component of multiple schedules on responding maintained by schedules of positive or negative reinforcement in an unchanging, constant component. Thus, in the case of positive contrast, broadly speaking, these changes involve what might be described as a "worsening" or degrading of the conditions of reinforcement in the changing component. For example, decreases in reinforcement rate or removing the reinforcer entirely (extinction), (signaled) delays of reinforcement (Richards, 1972), and (under some conditions) punishing responding (Brethower & Reynolds, 1962; but see Crosbie, Williams, Lattal, Anderson, & Brown, 1997) all increase response rates in the constant component. Conversely, a "bettering" or enhancing of conditions in the changing component by increasing reinforcement rate, decreasing delay, or reducing or removing punishment in the changing component often decreases response rates in the constant component, although such negative contrast effects do not always mirror positive ones in terms of the extent of the changes in responding (e.g., Marcucella & MacDonall, 1977).

Other experiments have examined the conditions maintaining responding in the constant component as a factor in the generation of behavioral contrast. For the most part, responding in that component is maintained by some schedule of positive reinforcement, often (but not necessarily) the same type as in the other component before the conditions in the latter are changed (e.g., a multiple variable-interval [VI] VI or multiple differential-reinforcement-of- low- rate [DRL] DRL schedule). Wertheim (1965) also reported behavioral contrast with rats when responding in the constant component was maintained by a free-operant avoidance procedure as a function of changes in response-shock intervals in the variable component and Lattal (1970) found both positive and negative contrast when each response in the constant component was punished by brief electric shock.

Another way of degrading reinforcement is to delay its presentation from the response that produces it. Richards and Marcattilio (1978), for example, maintained responding of pigeons on a multiple VI 60-s VI 60-s schedule. In the constant component, reinforcement was preceded by a 10-s or 20-s delay from the response that produced it. Each delay was accompanied by a stimulus change (blackout or turning on an additional stimulus light). For different groups, the delays were either of fixed or variable duration, the latter averaging the specified delay value. In the other, changing component, in the presence of a different colored key light, responding was first reinforced according to the same VI schedule as in the constant component except that each scheduled reinforcer was delivered immediately following the response that produced it. When the VI schedule in the changing component was replaced by extinction, mean response rates in the constant component, where reinforcement was delayed by 10 s, increased relative to baseline response rates in that component, toward the end of the condition. If the delay was 20-s, however, mean response rates in the constant component decreased during at least some of the sessions relative to the baseline response rate in that component. Whether the delays were fixed or variable made no difference. When the VI schedule was reinstated in the changing component, response rates in the constant component decreased in the case of the 10-s delay condition. Individual pigeon data across sessions were not presented; rather, all analyses were based on a be-tween-group statistical analysis of mean data. Thus, it was difficult to assess the reliability of the effect across individual subjects. More importantly, using a signal during the delay makes it difficult to determine whether the responding in the constant component was the result of a delay per se because stimuli correlated with delays of reinforcement may assume multiple functions in relation to the reinforcer (Lattal, 2010).

A truer test of behavioral contrast of responding maintained by delayed reinforcement requires eliminating the confounding effects of the signal. Thus, the present experiment examined the occurrence of behavioral contrast when responding was maintained by an unsignaled delay of reinforcement procedure in which responses initiated delays of reinforcement, but without any stimulus change associated with the delay. In addition, unlike Richards and Marcattilio (1978), individual subject data were analyzed and three extinction conditions were examined: either long- or short-duration blackouts designed to minimize responding in extinction, or a change of response-key color during the extinction component.

 

Method

Subjects

Three mature male White Carneau pigeons were maintained at 80% of free-feeding weight. Each had a history of responding on various schedules of reinforcement. One of them (1874) died before completing the last condition of the experiment. Each was housed individually in a vivarium with continuous access to water and health grit.

Apparatus

An operant conditioning chamber 30 cm wide by 30 cm high by 32 cm deep was used. One wall of the chamber contained a single 2.5-cm diameter response key, located on the midline. A minimum force of 0.15 N was required to operate the key. The key could be transilluminated red or green. The same wall also housed a houselight in the lower right corner that was transilluminated white throughout each session, except during reinforcement and scheduled blackouts (see Procedure section below). Reinforcement was 3-s access to mixed grain made available from a food hopper located behind a 5 cm square aperture located on the midline of the wall, with its lower edge 8 cm from the floor. The aperture was illuminated when the hopper was raised. White noise and a ventilating fan masked extraneous sound. Programming and recording equipment were located in an adjacent room.

Procedure

Each pigeon first was trained to peck in the presence of a red keylight according to a tandem variable-time (VT) 50-s fixed-interval 1 -s schedule of reinforcement. The VT schedule was composed of 20 intervals drawn from the distribution described by Flesh-ler and Hoffman (1962). When an interval lapsed, a 1 -s interval started. After that interval lapsed, the next response produced 3-s access to mixed grain. When responding of each pigeon stabilized, a tandem VI 50-s fixed-time (FT) 1-s schedule was introduced. Under this schedule, the first response after an interval drawn from the VI distribution lapsed initiated a 1 -s interval, at the end of which the reinforcer was presented. This tandem schedule defines a nonresetting, unsignaled delay of reinforcement procedure in that responses during the FT are without effect and the reinforced response is separated from the reinforcer by the FT duration. During this and all conditions that followed, each session ended after 90 reinforcers were delivered. After several sessions of responding at high rates with the 1-s delay in effect by Pigeons 3344 and 5511, the value of the FT was increased to 2 s for both. Responding then was stabilized before implementing the conditions described in the next paragraph. In this and all subsequent conditions, stable responding was defined by dividing the last 6 sessions into two 3-session blocks. When the mean response rate for both blocks did not differ by more than +/- 3% from the 6-day mean, responding was considered stable.

After obtaining stability on the tandem VI 50-s FT x-s schedule, a series of multiple schedule conditions were alternated with the tandem schedule, as shown in Table 1. One component, the constant component, of the multiple schedule was the tandem VI 50-s FT x-s schedule. In the other, changing component, responding was never reinforced under three different stimulus conditions, collectively described hereafter as extinction conditions. The extinction component was associated with a 20-s blackout (during which time the houselight and keylight in the chamber were turned off), a 100-s blackout, or 20-s period during which the response key was transilluminated green and the houselight remained on. Blackout and green-keylight periods occurred independently of responding and were presented at variable intervals averaging 50 s, with a range of 5 -180 s. In the final condition for Pigeons 3344 and 5511, a tandem VT 50-s FI 2-s schedule was in effect.

 

Results

Table 1 also shows the stable mean response rates of each pigeon during each condition and also the mean response rates averaged over the first and last six sessions of each extinction condition. The final unsignaled delay values (1 or 2 s) reduced each pigeon's stable response rates by 32-79% relative to the immediate reinforcement condition. For Pigeons 1874 and 3344, the unsignaled delays continued to maintain these relatively low rates across successive baseline conditions. The response rates of Pigeon 5511, however, gradually drifted upward across these successive baseline conditions. This drift is not surprising, given the contingencies on responding: if response rates decrease beyond a point, reinforcement rate can decrease; however, response rate increases have no adverse effect on reinforcement rate. Furthermore, given that extinction brings about response rate increases in the constant component, once such higher rate responding is reinforced, perhaps adventitiously, there is nothing to preclude the continuation of this high rate responding when extinction is removed.

The data in Table 1 show that each of the extinction procedures generated rates of responding between 2 and 21 responses per minute for different pigeons. Some of this responding, especially for Pigeon 1874, was "carry-over" responding that occurred immediately following the change from the constant component. For the other two pigeons, which had higher response rates in the extinction conditions, other processes, such as adventitious reinforcement of responding by the return of the stimuli correlated with the unsignaled delay component, may have been operative.

The effects of these extinction procedures on responding in the unsigned delay component are summarized graphically in Figure 1, where response rates in each of the extinction conditions are shown as a percentage change from the average of the immediately preceding and following baseline condition, where the extinction condition was absent (see Table 1). Response rates in the unsignaled delay component increased relative to the preceding no-extinction-present baseline condition when the extinction component was in effect. With some pigeons and some extinction conditions, these increases were greater in the first six sessions following introduction of the extinction component, but with other pigeons and extinction conditions the highest response rates occurred at the end of the condition. Across pigeons, too, the response rate increases were not related systematically to either the presence or absence of the keylight or to delay duration, although for Pigeons 1874 and 5511, the 100-s blackout engendered higher response rates in the constant, unsignaled-delay, component during the last six sessions of that condition.

When the extinction component was removed, with Pigeons 1874 and 3344 response rates in the unsignaled delay component consistently decreased, reversing toward the levels observed in the condition prior to introducing that extinction condition - a negative contrast effect. Pigeon 5511 also showed such a reversal after the first extinction condition. After the second extinction condition, however, its response rates in the unsignaled delay component drifted upward, failing to reverse to those of the previous baseline. After the third extinction condition, its mean response rates reversed, but they still were higher than those observed in the first unsignaled delay baseline condition. These response rates of Pigeon 5511 in the last tandem VI 50-s FT 2-s were identical to those maintained by the immediate reinforcement condition programmed in the last condition of the experiment. By contrast, Pigeon 3344's response rates increased by 80% when immediate reinforcement was reinstated (as noted above, Pigeon 1874 died before this condition could be introduced).

As the data in Table 1 show, mean reinforcement rates remained approximately constant in the presence of the red light when the delay of reinforcement condition was in effect.

 

Discussion

Positive behavioral contrast occurred during each extinction condition in the changing component. In only one instance did mean response rates in the constant component fail to reverse (negative contrast). Thus, contrast effects obtained consistently when responding was maintained under a VI schedule where reinforcement always was preceded by a nominal 1- or 2-s unsignaled delay. These results confirm Richards and Marcattilio's (1978) finding of positive behavioral contrast with responding maintained under a VI schedule where reinforcement was preceded by a 10-s signaled delay. Unlike the Richards and Marcattilio experiment, however, in the present experiment delays were not potentially confounded by other functions of signals accompanying them.

Richards and Marcattilio (1978) did not report actual response rates with immediate and 10-s signaled delayed reinforcement, stating only that "throughout training, subjects responded faster during immediate reinforcement than delayed reinforcement components and faster during delayed reinforcement than extinction components" (p. 60). Richards (1981) reported little reduction in responding with 10-s signaled delays. The data in Figure 1 from that experiment show that two of the pigeons had equivalently low response rates under 10-s signaled delays and 1-s unsig-naled delays. To an approximation, then, Richards's data suggest that 10-s signaled delays are functionally similar to much shorter unsignaled delays - delays in the range of the present unsignaled delay values of 1 - 2 s. Thus, Richards and Marcattilio's finding of behavioral contrast when 10-s signaled delays preceded reinforcement in the constant component is corroborated with shorter unsignaled delays in the present experiment that may have functionally similar effects to those longer signaled delays on response rates.

Richards and Marcattilio (1978) found no systematic evidence of positive contrast when 20-s signaled delays occurred before each reinforcer in the constant component when responding was extinguished in the changing component. They also did not report response rates maintained by the 20-s delays, however, nor did they report whether these rates differed from those when 10-s delays were in effect in the constant component. Both the present experiment and previous ones suggest that behavioral contrast can obtain even if response rates are relatively low. Pigeon 1874, for example, showed consistent contrast effects in the constant component even though its response rates were reduced by imposing a 1-s unsignaled delays by 79% from those obtained with immediate reinforcement. Lattal (1970) found behavioral contrast when responding in the constant component was reduced by 56 and 95% of its unpunished rate for the two pigeons studied, and Reynolds and Catania (1961) found behavioral contrast when constant component responding was maintained by differential-reinforcement-of-low-rate schedules. Richards and Marcattilio's data nonetheless raise the possibility that responding maintained at different rates may be differentially sensitive to reinforcement rate changes in the changing component. However, neither their data nor those from other experiments showing behavioral contrast with low rates are sufficiently complete to yet disentangle this relation.

Introducing a delay between a response and the reinforcer it produces is but one way of degrading the response-reinforcer relation. Another way of doing so is to eliminate the response-reinforcer dependency, thereby delivering rewards independently of responding. When the response-reinforcer relation is eliminated, responding previously maintained by immediate reinforcement typically decreases and is often eliminated (Herrnstein, 1966; Zeiler, 1968). Two experiments have reported contradictory findings with respect to whether behavioral contrast occurs in a constant component when rewards are delivered independently of responding in that component. Gamzu and Schwartz (1973) reported positive contrast with pigeons when responding in the constant component was maintained by a VT schedule of food delivery in which the food occurred independently of whether the pigeon responded on the operandum. Removing food in the other component reliably increased the rate of key pecking in the (constant) VT component. Bradshaw, Szbadi, and Bevan (1978), however, failed to replicate with rats the findings of Gamzu and Schwartz, even though when Bradshaw et al. maintained responding of the same rats with VI schedules and then extinguished responding in the changing component, response rates in the constant component increased, thereby evincing positive behavioral contrast. Because responding maintained by VT schedules is both variable and transient (potential species and other differences between Gamzu & Schwartz and Bradshaw et al. aside), it is not surprising that some variability in contrast effects, along with response rates themselves, is observed under such conditions. Whether response rates per se or some other variable was responsible for the differences reported in these two experiments cannot be discerned, leaving still unanswered the question of how constant-component response rates are implicated in the occurrence or failure of behavioral contrast. Unsignaled delayed reinforcement may be one useful way of controlling response rates in the constant component to further investigate the relation suggested by the Richards and Marcatillio (1978) data.

Systematic differences in positive contrast did not occur in this experiment as a function of the type of stimuli correlated with or the duration of the extinction component. Sadowsky (1973) reported qualitiative differences in behavioral contrast with pigeons as a function of associating a blackout versus a keylight color change with extinction in the changing component. Blackouts resulted in a rather constant increase in responding in the constant component that sustained as long as the blackout was in effect. A keylight color change accompanying extinction resulted in transient contrast, where responding increased abruptly with the change to extinction in the changed component and then decreased across successive sessions. In contrast, the data in Figure 1 reveal a mix of transient and sustained contrast, that is, across conditions and across pigeons, both the type and extent of contrast was not related systematically to the types of stimuli present during extinction. Sadowsky attributed the qualitative differences in contrast that he observed to the presence or absence of responding during extinction. In this experiment, however, both of the blackout conditions and the occurrence of a green keylight in extinction resulted in low response rates (the highest of these occurred for Pigeons 3344 and 5511 during the 100-s blackout condition, where response rates averaged 15-17 responses per minute). It could be that these low rates were due to the present pigeons' prior histories of exposure to reinforcement schedules and blackouts, whereas Sadowsky's pigeons were experimentally naive prior to his research. In only once instance (Pigeon 5511 under the 100-s blackout condition) did sustained contrast occur. Higher response rates during the first six sessions relative to the last six sessions of extinction in the changing component were observed several times, as were instances where response rates increased from the first to the last six sessions of an extinction condition.

Behavioral contrast was first described more than fifty years ago by Reynolds (1961). Its significance then and now was that targeted responding maintained by any contingency was affected not only by the nominally maintaining contingency, but also other contingencies that were proximal to it. It thus became necessary to consider targeted responding in a broader environmental context. Research on behavioral contrast has focused largely on these broader environmental contexts as determinants of behavioral contrast. The present research, along with the other experiments discussed herein, underline the potential importance of considering both context and current contingencies on the target response as sources of behavioral contrast.

 

References

Bradshaw, C. M., Szbadi, E., & Bevan, P. (1978). Behavior of rats in multiple schedules of response-dependent and response-independent food presentation. Quarterly Journal of Experimental Psychology, 30, 133-139. doi:10.1080/14640747808400661.         [ Links ]

Brethower, D. M., & Reynolds, G. S. (1962). A facilitative effect of punishment on unpunished behavior. Journal of the Experimental Analysis of Behavior, 5, 191-199. doi:10.1901/jeab.1962.5-191.         [ Links ]

Crosbie, J., Williams, A. M., Lattal, K. A., Anderson, M. M., & Brown, S. (1997). Schedule interactions involving punishment with pigeons and humans. Journal of the Experimental Analysis of Behavior, 68, 161-175. doi:10.1901/jeab.1997.68-161.         [ Links ]

Fleshler, M., & Hoffman, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5, 529-530. doi:10.1901/jeab.1962.5-529.         [ Links ]

Gamzu, E., & Schwartz, B. (1973). The maintenance of key pecking by stimulus-contingent and response-independent food presentation. Journal of the Experimental Analysis of Behavior, 19, 65-72. doi:10.1901/jeab.1973.19-65.         [ Links ]

Herrnstein, R. J. (1966). Superstition: A corollary of the principles of operant conditioning. In W. K. Honig (Ed.), Operant behavior: Areas of research and application (pp. 33-51). New York: Appleton-Century-Crofts.         [ Links ]

Lattal, K. A. (1970). Relative frequency of reinforcement and rate of punished behavior. Journal of the Experimental Analysis of Behavior, 13, 319-324. doi:10.1901/ jeab.1970.13-319.         [ Links ]

Lattal, K.A. (2010). Delayed reinforcement of operant behavior. Journal of the Experimental Analysis of Behavior, 93, 129-139. doi:10.1901/jeab.2010.93-129.         [ Links ]

Marcucella, H., & MacDonall, J. S. (1977). A molecular analysis of multiple schedule interactions: Negative contrast. Journal of the Experimental Analysis of Behavior, 28, 71-82. doi:10.1901/jeab.1977.28-71.         [ Links ]

Reynolds, G. S. (1961). Behavioral contrast. Journal of the Experimental Analysis of Behavior, 4, 57-71. doi:10.1901/jeab.1961.4-57        [ Links ]

Reynolds, G. S., & Catania, A. C. (1961). Behavioral contrast with fixed-interval and low-rate reinforcement. Journal of the Experimental Analysis of Behavior, 4, 387391. doi:10.1901/jeab.1961.4-387.         [ Links ]

Richards, R. W. (1972). Reinforcement delay: Some effects on behavioral contrast. Journal of the Experimental Analysis of Behavior, 17, 381-394. doi:10.1901/jeab. 1972.17-381.         [ Links ]

Richards, R. W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35, 145-152. doi:10.1901/ jeab.1981.35-145.         [ Links ]

Richards, R. W., & Marcattilio, A. J. (1978). Stimulus control and delayed reinforcement. Learning and Motivation, 9, 54-68. doi:10.1016/0023-9690(78)90026-7.         [ Links ]

Sadowsky, S. (1973). Behavioral contrast with timeout, blackout, or extinction as the negative condition. Journal of the Experimental Analysis of Behavior, 19, 499-507. doi:10.1901/jeab.1973.19-499.         [ Links ]

Wertheim, G. A. (1965). Behavioral contrast during multiple avoidance schedules. Journal of the Experimental Analysis of Behavior, 8, 269-278.         [ Links ]

Zeiler, M. D. (1968). Fixed and variable schedules of response-independent reinforcement. Journal of the Experimental Analysis of Behavior, 11, 405-414.         [ Links ]