The Brightness Lab module integrates math and science curricula in a study of brightness perception. The module begins by addressing misconceptions about vision and the perception of visual illusions. After an intuitive and qualitative presentation of brightness perception through physical models similar to actual scenes, eye and brain anatomy are presented. The module gradually introduces quantitative models, beginning with ratios, to explain the phenomenon and deepen understanding. Thus, models range from those similar to everyday experience to increasingly abstract concepts and mathematical formalizations.

The specific topic in this module is the brightness contrast effect wherein squares of the same shade of gray appear to be different when embedded in different backgrounds. The first part of the software interaction serves to clarify the phenomena in concrete terms. A split screen is presented with the left side of the screen containing “reference” concentric squares and the right side an inner “target” square within an outer square (see Figure 1).

Figure 1: The surrounding context impacts our perception of brightness. The reference and the target inner square patches are the same shade of gray.

The reference stimulus presents the inner and outer squares filled-in with different but fixed gray colors. The target square is also gray, but the grayscale is adjustable by the student. The student must adjust the shade of gray of the target square to match the appearance of the color-fixed inner reference square. A series of reference stimuli are presented randomly from an already-prepared set of reference colors. Once the student perceives a match, the student instructs the computer to record his/her choice of the color and continue to the next reference item to match. This is repeated a number of times over a set of reference colors. Each student records a luminance measurement of the reference and matched target colors and computes their ratio. At the end of a session the student is provided with a chart of the results of the session. Depending on the level of the course of study, the student will graph and analyze the different variables in the experiment to discover patterns and reach conclusions that will allow them to explain brightness perception and make predictions about brightness contrast and constancy.

Thus, building on an intuitive sense of brightness perception, the student discovers various properties about it through interactive hands-on experiences. The module presents a sequence of concrete and abstract models that represent and explain key phenomena (for example, brightness contrast and brightness constancy [7]), and are structured towards progressive formalization of knowledge. The mathematical models range from (1) simple ratios of measures of reflected light to illustrate brightness contrast to (2) complex fractions that explain brightness constancy, to (3) scatter plots and (4) linear and nonlinear regression analysis. Students are introduced to concepts from discrete mathematics such as (5) directed graphs to design a neural network diagram. This exercise leads to the (6) representation of the dynamic values of the nodes in the network that represent the biological substrate for vision and influences on their rates of change. Then, (7) equilibrium analysis using simple algebra can be performed (the equation for the rate of change is set to zero) to study the properties of a mass-action neural network model in terms of its parameters and variables; finally, (8) calculus notation can be introduced. The level and degree of math used in the model is tailored to various grade levels; see Figures 2a and 2b for examples of the mathematics of brightness perception suitable for a short lesson or “stealth module” for algebra students.

Figure 2a: From image to ratio: an example of using the Brightness Lab as a “stealth module” in elementary algebra class. Brightness perception is based on comparing the luminance values of two surfaces; the illuminant is not relevant.

Figure 2b: Brightness perception illusions are based on the brain’s ability to calculate local contrasts and compare them across the visual field. Since Contrast A > Contrast B, it appears brighter.

There are two areas that call for a problem-solving approach to mathematics. First, integrated with use of the Brightness Lab software, a formal lab report helps students structure and assess their thinking as they collect experimental data about themselves, create visual displays of the quantitative data, and perform statistical analysis and interpretation. Second, a neural network model of brightness perception (see Figure 3) uses an ordinary differential equation to characterize the rate of change in neurons that leads to brightness perception.

Figure 3: Software for student inquiry about brightness perception uses simulations based on a mass-action, shunting neural network. The equation represents the rate of change of neuronal activity.

Students are guided to the construction of the neural network model using an introduction to terminology, graphing symbols, and typical neural interactions of excitatory and inhibitory connections. Directed experiments with simulations using software layers (controlled by the tabs on the top of the screen) are also provided. Parameters can be adjusted for simulations using drop down boxes or scroll bars on left of screen. The user can vary both the input as well as the process parameters of the network (the rate of passive decay of the signal, and maximum levels of excitation and inhibition in the system).

The structure of the software interaction is based on layers, or more concretely, on “tabs” within the program. In this way, the first layer is presented in the experiment and analysis tabs. A second layer is planned to take the student to an anatomy lesson (this part has been completed in print form only). The third layer takes the student to different aspects of modeling. Depending on the grade level in which the module is being used, different tabs or layers will be active, showing more (or less) information as appropriate to that particular grade. The software provides the interactivity and feedback needed to grasp a working knowledge of the brightness contrast effect. With deep understanding, the student will demonstrate transfer of knowledge to other contexts, such as art appreciation [8] (see Figure 4), graphic design, or night driving.

Figure 4: The bright-dark (chiaroscuro) technique achieves multiple aesthetic effects in The Girl with the Pearl Earring by Johannes Vermeer.

Feedback from teachers during the 2007 Summer Workshop has helped to refine the modules. For example, when secondary math teachers said, “Too much science.”, the “For Math Class” Workbook was revised to focus on the review and application of linear regression analysis. Several responses to feedback that “The neural network is too abstract.” are currently in progress. An exciting change is a modification of the software so the computer simulates the brightness matching game to tie the neural network model directly to the interactive experience of data collection experiment. Furthermore, an experiment history screen will help students (1) compare lines and nonlinear curves of best fit, (2) discuss how and why adjusting model parameters for passive decay, input-based excitation, and competitive interaction impacts the regression equations based on the computer simulations, and (3) compare these simulations with their personal data. To help the reader understand the derivation and structure of the neural network model of the perception of brightness contrast, the curriculum is currently being revised to include a more qualitative discussion about issues the visual system must resolve in order to process a wide range of input without loss of sensitivity. Finally, it is anticipated that presenting the quantitative model with reduced complexity (removing summation notation and subscripts) will help to focus attention on how the interaction of contrasting surfaces relates to the subjective experience of brightness perception.

The Sequence Learning module integrates math and science curricula to study memory related to temporal order processing. Furthermore, the module is designed to develop study skills and introduce the scientific method via a formal lab report. Students collect experimental data on their own working memory span, perform an analysis thereof, and test the negative effect of distractions and the positive effect of explicit chunking on their memory span. There is also an opportunity to use an advanced mathematical model of sequence learning. Students discover that attention is a key factor in learning and that an individual’s memory capability varies and can be improved with effort.

After students have participated in short experiments and discussion aimed at showing how important yet fragile their working memory is, and the typical features of working memory performance (the first few and most recent items in a sequence are usually remembered best), they will be guided through the Sequence Learning software. The software module implements a set of psychophysical experiments that allow data to be collected to illustrate properties of working memory: list length/span (immediate serial order recall for lists of tokens such as digits, letters or pictures of varying lengths); list serial position performance (or error) curves showing better recall performance of items at the beginning and sometimes at the end of the list (at span and span+1); and list serial position performance curves with grouping. The software is easy to use with an intuitive interface, embedded instructions, and context sensitive help. The software supports the quantitative assessment of aspects of working memory that are explained in the background and theory document. Moreover, there is a seamless integration of the psychophysical experiment with the model simulation.

To begin the module, students are invited to consider their own experience and ask questions about possible mechanisms and models of sequence learning. In a motivating activity, students are confronted with a classic example of working (short-term) memory: remembering a phone number that you were just told. However, this type of memory is extremely fragile (i.e. easy to forget). For instance, if you hear the telephone number but accidentally trip on the way to dialing it, or if someone comes up to you and asks for directions immediately after hearing it, then you may not be able to dial it after you stand up again or answer the person's question; being distracted can greatly diminish your ability to hold things in working memory. In contrast, even if you are distracted, you can still recite your own name or your own telephone number when you need to. This sort of information is more stable and can be remembered virtually whenever you want; it is recalled from long-term memory. A natural question that arises for the students is: What properties of our working memory enable us to keep short sequences of events "in our minds" for a few seconds, but also allow us to learn particular, important sequences in a more stable way?

Module activities in the class include consideration of the issue: why can't we store arbitrarily long sequences in working memory? The question of why is there a natural upper bound on how many items can be stored at the same time [9] is explored in the software component for span testing. Students perform exercises where they generate the primacy effect where the initial items in a list are remembered better than later items (see Figure 5). The primacy effect is the first part of the common inverted-U performance profile related to serial position during free recall of the temporal order of list items. Students learn that this limitation reflects how the specific neural activities related to item-and-order memory change when new items come in. However, with longer lists there is also a recency effect where the last items in a list are remembered better than pervious ones. The recency effect completes the inverted-U performance profile and implies that that the items do not get stored with a primacy gradient all the time. Why is this? To have a long-term memory that retains a true representation of order information, we can only store relatively short lists in working memory at any given time. To be established in long-term memory, the short lists must typically be rehearsed. Longer lists in working memory would cause an inverted-U that would be transferred to long-term memory. Therefore, working memory and long-term memory are specialized such that they complement each other to provide rapid new learning along with stable old learning. Students are encouraged to develop these conceptual models of how our brains may be storing sequences of information in a working memory. One important constraint on how brains perform this task is that whenever we are actively thinking about an item in working memory, there is a specific location or network in the brain where that item is being represented. At least two possibilities exist for theories of how items in a sequence are stored within our working memory: positional models [10] and ordinal models [11].

Figure 5: This visual display of psychometric data shows a clear primacy effect.

Students are guided through the construction of the neural network model using an introduction to terminology, graphing symbols, and typical neural interactions of feed-forward and feedback excitatory and inhibitory connections. Directed experiments with simulations using the model layer are provided on two levels. Level 1 is a conceptual use of the model, and simulations are comprised of the manipulation of three parameters (bottom-up activation, top-down expectation and attention) to see how these impact the ability of the model to learn a given sequence of digits (see Figure 6). Level 2 includes a complex network diagram, and the actual differential equations that support the outcomes of the model; 6 parameters are exposed. The Sequence Learning neural network model that is active in the software is an ordinal model called LISTPARSE (Laminar Integrated Storage of Temporal Patterns for Associative Retrieval, Sequencing and Execution [12]) which explains and quantitatively simulates cognitive data about immediate serial recall and free recall, including bowing of the serial position performance curves, error-type distributions, limited temporal extent for accurate recall, and word and list length effects.

Figure 6: Software support for inquiry using a neural network simulation of sequence learning.

A valuable transfer of the lessons of this module is an application of the knowledge of the primacy/recency effects by both teachers and students for improved pedagogy and learning. To improve educational effectiveness over a given duration, activities should be segmented to short time-frames to maximize the number of primacy/recency events and thereby allow different learning objectives to enjoy the enhanced learning structure. In addition, if curriculum content is randomly practiced, there will be more effective long-term learning compared with routine, blocked drill because different items would enjoy the boost provided by primacy/recency positioning.

Temporal order processing also occurs with action sequences, such as pitching or hitting a baseball, playing a musical instrument, typing, or dancing. At first, you have to focus on each individual movement in the sequence and performance is irregular, but eventually you can perform an entire group of movements (or motor sequences) smoothly together. Were it not for working memories that enable us to learn these “sequence chunks”, we would have to spend the whole day rehearsing the simplest activities, so as not to forget them; e.g., tying your shoes, brushing your teeth. Having a working memory that can allow chunks to be learned enables us to focus our attention on more interesting aspects of life, and is a crucial foundation for society, culture, and creativity. We are currently working with CELEST partners at Brandeis University to implement research on a stroke-imitation task [13] into the module. In this way, the user of the module will be presented with both cognitive and motor tasks that illustrate the properties of sequence learning.

Associative learning is a basic form of animal and human cognition; adaptive-timing is a basic form of associative learning. We can easily recognize that most human brain activity and behavior require precise timing: for example, sleep, perception, speech, writing, walking, running, playing sports and video games of all types; you can think of several more examples. The ability to learn temporal conditioning is a critical survival competence in normal adaptive behavior because it enables the learning of which earlier events predict later consequences, as well as which event combinations are not causative. In this way, the individual can make the optimal choices for successful, adaptive behavior.

This module explores two conditioning paradigms for adaptive-timing as representative of procedural and declarative memory, namely, delay and trace conditioning.  The learning measured is the association of a heretofore neutral event, such as a tone or a light, with an emotionally-charged, reflex-inducing event, such as a puff of air to the eye or a shock to some part of the body. The first event is called the conditioned stimulus (CS); the second is the unconditioned stimulus (US).  Delay conditioning is said to occur when the stimulus events have temporal overlap and coterminate so that the subject associates concurrent sensations and learns to make an adaptive, conditioned response (CR) in anticipation of the US. Trace conditioning involves a temporal gap between the CS offset and the US onset such that a memory trace is required to make a successful CR.

In the first level of the module, students begin with an experimental, interactive task that simulates aversive conditioning: click the mouse before a red disc (the US) appears. There are multiple training schedules of the appearance of the red disc: (1) a random appearance, (2) at a fixed time interval as an event that follows an initial blue disc (CS), and (3) a probabilistic appearance at a fixed interval after the CS terminates. In the second level, the visual display of results for each schedule support a discussion of conditioning, adaptive timing and the learning curve; students discover their own learning curve in the fixed schedule trace paradigm. Performance data is presented in multiple formats to show their differences: bar graphs, scatter plots, line graphs, and, to begin a higher level of analysis, linear regression. We are currently enhancing the software to include nonlinear models of the learning curve that may have a better fit to the data.

The third level of the module uses the study of eye-blink conditioning to develop a deeper understanding of the types of memory. The eyeblink response, measured by eyelid movement in rats, cats, monkeys and humans, and the movement of the nictitating membrane (NM) that covers the eye like an eyelid in rabbits, has been extensively studied because it has given excellent parametric behavioral, neurophysiological and anatomical information about the learning and memory processes related to associative conditioning. There is a complete model layer based on published research [14] that simulates the NM response as measured by the timing and amplitude of its movement and the cerebellar response that supports the behavior. The software simulation is comprised a multi-dimensional GUI interface with adjustable parameters such as the conditioning paradigm, the onset/offset times of CS and US, the learning and extinction rates, and number of Purkinje cells in the network (see Figure 7).

Figure 7: GUI support for inquiry about behavioral and cellular responses using a neural network simulation of adaptive timing.

There are currently three phases in this level, each exposing the model and the simulation results in greater detail. Similar to the effort in Brightness lab, work is underway to provide a simulation of the experimental task related to the blue and red discs performed by the students in this module so there will be a tight coupling of the neural network to student experience. We have found this approach effective in Sequence Learning.