A theoretically motivated method for automatically evaluating texts for gist inferences Abstract We developed a method to automatically assess texts for features that help readers produce gist inferences. Following fuzzy-trace theory, we used a procedure in which participants recalled events under gist or verbatim instructions. Applying Coh-Metrix, we analyzed written responses in order to create gist inference scores (GISs), or seven variables converted to Z scores and averaged, which assess the potential for readers to form gist inferences from observable text characteristics. Coh-Metrix measures reflect referential cohesion and deep cohesion, which increase GIS because they facilitate coherent mental representations. Conversely, word concreteness, hypernymy for nouns and verbs (specificity), and imageability decrease GIS, because they promote verbatim representations. Also, the difference between abstract verb overlap among sentences (using latent semantic analysis) and more concrete verb overlap (using WordNet) should enhance coherent gist inferences, rather than verbatim memory for specific verbs. In the first study, gist condition responses scored nearly two standard deviations higher on GIS than did the verbatim condition responses. Predictions based on GIS were confirmed in two text analysis studies of 50 scientific journal article texts and 50 news articles and editorials. Texts from the Discussion sections of psychology journal articles scored significantly higher on GIS than did texts from the Method sections of the same journal articles. News reports also scored significantly lower than editorials on the same topics from the same news outlets. GIS proved better at discriminating among texts than did alternative formulae. In a behavioral experiment with closely matched text pairs, people randomly assigned to high-GIS versions scored significantly higher on knowledge and comprehension.

The influence of place and time on lexical behavior: A distributional analysis Abstract We measured and documented the influence of corpus effects on lexical behavior. Specifically, we used a corpus of over 26,000 fiction books to show that computational models of language trained on samples of language (i.e., subcorpora) representative of the language located in a particular place and time can track differences in people’s experimental language behavior. This conclusion was true across multiple tasks (lexical decision, category production, and word familiarity) and provided insight into the influence that language experience imposes on language processing and organization. We used the assembled corpus and methods to validate a new machine-learning approach for optimizing language models, entitled experiential optimization (Johns, Jones, & Mewhort in Psychonomic Bulletin & Review, 26, 103–126, 2019).

How well do word recognition measures correlate? Effects of language context and repeated presentations Abstract In the present study we assessed the extent to which different word recognition time measures converge, using large databases of lexical decision times and eyetracking measures. We observed a low proportion of shared variance between these measures, which limits the validity of lexical decision times to real-life reading. We further investigated and compared the role of word frequency and length, two important predictors of word-processing latencies in these paradigms, and found that they influenced the measures to different extents. A second analysis of two different eyetracking corpora compared the eyetracking reading times for short paragraphs with those from reading of an entire book. Our results revealed that the correlations between eyetracking reading times of identical words in two different corpora are also low, suggesting that the higher-order language context in which words are presented plays a crucial role. Finally, our findings indicate that lexical decision times better resemble the average processing time of multiple presentations of the same word, across different language contexts.

Improving the performance of eye trackers with limited spatial accuracy and low sampling rates for reading analysis by heuristic fixation-to-word mapping Abstract The recent growth in low-cost eye-tracking systems makes it feasible to incorporate real-time measurement and analysis of eye position data into activities such as learning to read. It also enables field studies of reading behavior in the classroom and other learning environments. We present a study of the data quality provided by two remote eye trackers, one being a low-sampling-rate, low-cost system. Then we present two algorithms for mapping fixations derived from the data to the words being read. One is for immediate (or real-time) mapping of fixations to words and the other for deferred (or post hoc) mapping. Following this, an evaluation study is reported. Both studies were carried out in the classroom of a Finnish elementary school with students who were second graders. This study shows very high success rates in automatically mapping fixations to the lines of text being read when the mapping is deferred. The success rates for immediate mapping are comparable with those obtained in earlier studies, although here the data is collected some 10 min after initial calibration of low-sample (30 Hz) remote eye trackers, rather than a laboratory setting using high-sampling-rate trackers. The results provide a solid basis for developing systems for use in classrooms and other learning environments that can provide immediate automatic support with reading, and share data between a group of learners and the teacher of that group. This makes possible new approaches to the learning of reading and comprehension skills.

Quaddles: A multidimensional 3-D object set with parametrically controlled and customizable features Abstract Many studies of vision and cognition require novel three-dimensional object sets defined by a parametric feature space. Creating such sets and verifying that they are suitable for a given task, however, can be time-consuming and effortful. Here we present a new set of multidimensional objects, Quaddles, designed for studies of feature-based learning and attention, but adaptable for many research purposes. Quaddles have features that are all equally visible from any angle around the vertical axis and can be designed to be equally discriminable along feature dimensions; these objects do not show strong or consistent response biases, with a small number of quantified exceptions. They are available as two-dimensional images, rotating videos, and FBX object files suitable for use with any modern video game engine. We also provide scripts that can be used to generate hundreds of thousands of further Quaddles, as well as examples and tutorials for modifying Quaddles or creating completely new object sets from scratch, with the aim to speed up the development time of future novel-object studies.

Examining cross-level effects in dyadic analysis: A structural equation modeling perspective Abstract The actor–partner interdependence (APIM) and common-fate (CFM) models for dyadic data are well understood and widely applied. The actor and partner coefficients estimated in the APIM reflect the associations between individual-level variance components, whereas the CFM coefficient describes the association between dyad-level variance components. Additionally, both models assume that the theoretically relevant and/or empirically dominant component of variability resides at the same level (individual or dyad) across the predictor and outcome variables. The present work recasts the APIM and CFM in terms of dyadic nonindependence, or the extent to which a given variable reflects dyad- versus individual-level processes, and describes a pair of hybrid actor–partner and common-fate models that connect variance components residing at different levels. A series of didactic examples illustrate how the traditional APIM and CFM can be combined with the hybrid models to describe mediational processes that span the individual and dyad levels.

The Semantic Librarian: A search engine built from vector-space models of semantics Abstract Psychologists have made substantial progress at developing empirically validated formal expressions of how people perceive, learn, remember, think, and know. In this article, we present an academic search engine for cognitive psychology that leverages computational expressions of human cognition (vector-space models of semantics) to represent and find articles in the psychological record. The method shows how psychological theory can be used to inform and aid the design of psychologically intuitive computer interfaces.

A simple location-tracking app for psychological research Abstract Location data gathered from a variety of sources are particularly valuable when it comes to understanding individuals and groups. However, much of this work has relied on participants’ active engagement in regularly reporting their location. More recently, smartphones have been used to assist with this process, but although commercial smartphone applications are available, these are often expensive and are not designed with researchers in mind. To overcome these and other related issues, we have developed a freely available Android application that logs location accurately, stores the data securely, and ensures that participants can provide consent or withdraw from a study at any time. Further recommendations and R code are provided in order to assist with subsequent data analysis.

Randomized single-case AB phase designs: Prospects and pitfalls Abstract Single-case experimental designs (SCEDs) are increasingly used in fields such as clinical psychology and educational psychology for the evaluation of treatments and interventions in individual participants. The AB phase design, also known as the interrupted time series design, is one of the most basic SCEDs used in practice. Randomization can be included in this design by randomly determining the start point of the intervention. In this article, we first introduce this randomized AB phase design and review its advantages and disadvantages. Second, we present some data-analytical possibilities and pitfalls related to this design and show how the use of randomization tests can mitigate or remedy some of these pitfalls. Third, we demonstrate that the Type I error of randomization tests in randomized AB phase designs is under control in the presence of unexpected linear trends in the data. Fourth, we report the results of a simulation study investigating the effect of unexpected linear trends on the power of the randomization test in randomized AB phase designs. The implications of these results for the analysis of randomized AB phase designs are discussed. We conclude that randomized AB phase designs are experimentally valid, but that the power of these designs is sufficient only for large treatment effects and large sample sizes. For small treatment effects and small sample sizes, researchers should turn to more complex phase designs, such as randomized ABAB phase designs or randomized multiple-baseline designs.

Do complex span and content-embedded working memory tasks predict unique variance in inductive reasoning? Abstract Complex span and content-embedded tasks are two kinds of tasks that are designed to measure maintenance and processing in the working memory system. However, a key functional difference between these task types is that complex span tasks require the maintenance of information that is not relevant to the processing task, whereas content-embedded tasks require the maintenance of task-relevant information. The purpose of the present research was to test the hypothesis that more unique variance in inductive reasoning would be explained by content-embedded tasks than by complex span tasks, given that inductive reasoning requires reasoners to maintain and manipulate task-relevant information in order to arrive to a solution. A total of 384 participants completed three complex span tasks, three content-embedded tasks, and three inductive reasoning tasks. The primary structural equation model explained 51% of the variance in inductive reasoning; 45% of the variance in inductive reasoning was uniquely predicted by the content-embedded latent factor, 6% of the variance was predicted by shared variance between the content-embedded and complex span latent factors, and less than 1% was uniquely predicted by the complex span latent factor. These outcomes provide a novel extension to the small but growing literature showing an advantage of using content-embedded rather than complex span tasks for predicting higher-level cognition.