COGS 105: Research Methods for Cognitive Scientists

After gathering further information about you and your projects, we have decided to touch on two final topics that should be of great interest, and that you reported interest in: neuroscience and clinical psychology (or neuropsychology). We have identified some resources that will guide our discussion this week on these topics. The first is on an important project called the "Connectome Project." The second is a discussion of how neuroscience and cognitive science converge in the clinical domain.

Required reading #1. Very brief introduction to the Human Connectome Project.

Required reading #2. Some methodological details about the connectome.

Required reading #3. A discussion of how clinical neuropsychology works (consider pages 402 to 416; check out the subsequent emotion sections optionally).

Optional reading. What is the importance of scientific training if you are interested in clinical research? It is key. Consider this classic model in clinical psychology. It exemplifies how we combine these frames of thought (scientific, clinical).

Judging from our laboratory exercises, many of you have an interest in the design, implementation, and deployment of robotic systems from a cognitive perspective. Let's discuss this during the week.

Required reading #1 (short theory), required reading #2 (short popular), and required reading #3 (advanced example). So how would we design a robotic system that navigates the world and solves problems for us? There are two strategies that are sometimes regarded as competing with each other. One sees us as building a "CRUM-like" internal cognitive system and then testing its behavior in the world. Another is to build the robotic system while we track how it engages the world -- thus the world and how the robotic engages it are part of the design story. We will consider these strategies here.

Optional talk. A fascinating presentation on Google's driverless car (Prof. Thrun).
Optional reading. An update on autonomous transport vehicles from the proinent IEEE Robotics & Automation Magazine (free if you access it on campus!).

Let's consider another example of potential workplace excitement using cognitive science methods. We will have a special guest Shreya Gupta, who graduated with her COGS B.S. from UC Merced last year. She will explain UX Research and Design, and we'll further talk about the relationship between cognitive science methods and industry.

Required reading #1. This is a short and punchy statement and review of UX Research.

At UC Merced we are a hardcore cognitive science group, but the skills we convey in these basic introduction methods courses, and the skills we convey in our Ph.D. program, are amenable to different career paths. We will consider two of these this week. While it is important for us to consider theory and method in a pure form, understanding how these methods extend into the real world and careers is crucial. We will consider two advanced examples of this.

Required reading #1. The Compleat Academic is a famous book that graduate students are often told to read as they start their training as social or cognitive scientists. This is the first chapter, describing getting into a program and how to be successful in a program. Dreaming big? This is a kind of manual to those dreams. The whole book is great, check it out.

Required reading #2. "Industry" offers a domain in which to apply our scientific konwledge and methods. I put this term in quotes because, of course, there are many paths in industry. We will consider one that is growing rapidly, and is connected to our discussion this semester: data science. Once we have some knowledge of big data and data science skills, what do we do and where do we go? Let's consider some simple examples. Note: This reading expects you to have some understanding of SQL. Don't worry about this for now. It is mostly understandable from a general perspective, and showcases a problem-solving scenario in industry.

Optional readings. Here are some fun and intriguing readings on data and industry. Enjoy. From Mode Analytics: 4 Questions to Consider When Making Your Next Analytics Hire. Here's a brief blog post on Big Data fro Beginners. Does a Ph.D. trap you in academia? No. Here's an excellent lecture by Nadia Jaber on a new movement among many folks who go the Ph.D. route. Very interesting.

Neural network models can be contrasted, in an important sense, with a different kind of modeling strategy: using probabilities and something called "Bayes theorem" to predict cognition in very interesting (and sometimes very effective) ways. We'll cover that this week.

Required reading #1, required reading #2. Bayesian models are all the rage in cognitive science, and well beyond. This week we will consider how Bayes models work. The basic Bayes approach can be explained very simply in an elegant model from some lab friends at Stanford. We will consider this here.

Optional reading #1. The critique of Marcus and Davis above has been responded to in an exciting academic dialogue. Click on the optional reading to check it out.

Before moving into brain-imaging methods, let's talk a bit about how to model brains (very very loosely -- this point is controversial, to some extent). We'll start with some neural basics and move into the general neural-network approach. Neural networks are computer models that let us explore how parallel processing may work to generate cognition.

Required reading #1, required reading #2. For this week, let's focus exclusively on our own Prof. Jeff Yoshimi. He designed, programmed, and disseminated his very own neural-network framework called Simbrain. It's very easy to use and very dynamic and entertaining. Prof. Yoshimi has a nice writeup about neural networks which will be the basis for this week's class.

A lot of natural data / big data is simply in the form of human language -- and unsurprisingly, it is a wide ranging arena in which to test ideas about cognitive processing by using big-data-inspired methods. This week we will discuss two examples of this. One will be the creation of "models of meaning," and the other is to explore "linguistic fingerprints" that are controversial but quite interesting. Let's jump in.

Required reading #1. LSA = Latent Semantic Analysis. It is the basis of every Google search, just about. It is being used in intelligent conversational agents. It can be easily used by us to play with language! (Note this reading is quite length, but it is an easy read. We will, of course, not be able to cover all of it in class. Make sure to refer to the study guide prior to the exam.)

Required reading #2. LIWC = Linguistic Inquiry and Word Count. It is a controversial, but surprisingly effective, way to analyze patterns of meaning in text. We will discuss how LIWC works, how it has been used, and use it in lab. It also serves as a demonstration of the relationship between qualitative and quantitative methods. (Note: The reading is quite clear but lengthy, so we will not be covering all material in the one class. Focus on the first 10-or-so pages, which introduce LIWC and some core examples.)

The 21st-century is going to see a massive change in the ways that we can test theories in cognitive science. We're going to jump from cognitive processes all the way to "Big Data." User logs, Facebook posts, Instagram pictures, restaurant reviews, etc. -- these are now accessible to researchers for analysis, and can shed surprising light on culture and cognition. Let's consider a couple of examples this week. We will consider how you can access these data, and what you do with the data once you have them.

Required reading #1. Some of you may have seen this paper, but it is a demonstration of the cognitive and cultural questions we can ask with Google data. Google has digitized millions of books and -- get this -- Google has let anyone and their dad access the data and perform analysis on it.

Required reading #2. We can also turn science onto itself by studying publications in science. Google Books above has digitized 4% of all books ever printed. Some researchers have begun to take a similar Big Data strategy with published journal articles about the brain. We will learn about neurosynth and use it in class.

Optional readings: Check out this nice summary from the Association for Psychological Science.

Having covered priming, we are now in a perfect place to discuss research ethics. The past 5 years has seen an explosion in journals and in online discussions forums on the potential nonreliability of a lot of priming experiments. This is quite a controversial area, but you should be familiar with it. Let's discuss this and general research ethics, in particular two ideas: publication bias and the importance of replication.

Required reading. This paper summarizes the problem with some priming experiments, and gives an example of a groundbreaking large-scale study to replicate a series of experiments.

Optional material. Check out this intriguing and powerful special issue on these matters. Here is an interesting article on the matter, related to last lecture's primary author.

Reaction time is great, but it is not the only way we can collect behavioral data. Sometimes, we're just interesting in influencing people's decisions in the first place! Let's jump into the realm of social cognition, a very active research area filled with provocative findings and theoretical debate. Let's specifically talk about priming.

Required reading. This is a challenging 10-page reading, but it should be broadly comprehensible along with lecture. It contains a lot of detail and interesting examples.

Optional material. The core concept of priming can be considered here. Feel free to read this first, if you like. Optional.

Now that we've got some core concepts regarding sampling and measurement down, let's jump directly into methods related to cognitive science. The bread and butter of cognitive psychology has been the reaction-time (RT) experiment. Let's discuss methodological issues related to the collection and analysis of human behavior in reaction-time studies.

Required reading 1. First, read up on RT again (remember that 'mental chronometry' was included as an optional and interesting reading last week). This will give you some broader context on the importance of RT from a simple perspective, and set the stage for a more complex reading below. (We will discuss in class that this reading is a bit simplistic when it comes to certain things, such as the role of the left and right hemispheres, and the nature of individual differences in RT.)

Required reading 2. This is a challenging primary source reading, straight from a journal article on the relationship between processing speed and aging. You should find it interesting for a variety of reasons -- it relates RT and aging, first of all -- but it also detailed the issues that we face when doing research on this. This will be challenging to read, but I will guide us through key concepts in class.

Optional material. This paper is even more advanced than the one above, and is on quite a controversial topic, relating RT to intelligence. Enjoy. Here is another important recent paper, challenging this entire paradigm. Enjoy this too.

Studying behavior using observation and experiments is a key methodology of cognitive science, and has roots in psychology, especially cognitive psychology. Eventually, we will discuss experimental methodologies for the study of human behavior. We'll start with key methodology concepts of which we'll need a basic understanding.

The following readings are from the very useful Web Center for Social Research Methods. All online, all free. Enjoy!

Required reading 1. First, read this excellent summary of how we "sample" from people to collect data. These are very important principles to understand. The whole section is very useful.

Required reading 2. There are a number of other important concepts that you should have a basic understanding of. Feel free to focus on these pages primarily: Measurement Validity Types (under Construct Validity), Reliability and Validity (under Reliability), and Levels of Measurement.

Optional material. In lab you will be working on a reaction-time study and applying some of these concepts to a behavioral study you will design yourself. Feel free to read about the history of reaction time, under the austere heading of mental chronometry.

Philosophy involves a lot of sitting around, but don't let that fool you. There's lots of fun detail to what philosophers do. Let's check it out.

Required reading 1. Philosophers have helped us sort out the basic scientific method, and its logical properties. Let's read about it in this nice demo.

Required reading 2. One technique philosophers use is the development of scenarios or "thought experiments" that guide our thinking. They are powerful "intuition pumps" that are used to clear up conceptual confusions or arguments. Read about them here.

Optional material: Wikipedia has an amazing list of thought experiments. Which do you like?

Check it out: Famous philosopher Dan Dennett has recently come out with a book on thought experiments and other tools for thoughts. If you find this interesting, might be worth checking out!

Let's get thinking. How do we know each of us is a conscious creature? How could we tell if an artificial system is conscious, or thinking? These problems are real pickles. But pickles are delicious! Chew down on some of this classic discussion.

Required reading 1. Read and participate in this Turing Test module.

Required reading 2. As a refresher, read this summary of cognitive science. It is important to understand the subtle distinctions among the fields of cognitive science, and keep some of its core issues in mind.

Optional material: Read about the Loebner Prize, and try out some bots!