Instruments

All the participants completed an EI test of Chinese, a strategy use questionnaire, and a language background questionnaire. The EI test was designed based on a corpus analysis of the widely used Chinese textbooks, the primary source of the language input and use for L2 learners of Chinese in U.S. universities. According to a survey conducted to over 200 universities by the Chinese Language Teachers Association of the U.S. (Li et al., 2014), we compiled a corpus of 36 widely used Chinese textbooks, amounting to a total of 688 unit texts and 703, 995 characters. The corpus was divided into three sub-corpora, corresponding to the three proficiency levels of a typical university-level Chinese program, that is, beginning, intermediate, and advanced. Using corpus analysis techniques, we identified a list of commonly shared vocabulary and grammar (i.e., occurs the most frequently) across the textbooks at each of the three proficiency levels. Based on this list, we designed three sets of EI sentences, each set targeting one of the three proficiency levels. In addition, to reflect the language that L2 learners of Chinese are exposed to, the actual average length of the sentences at each level in the textbook corpus were adopted in the EI item design, which are 8 syllables (beginning and intermediate levels) and 12 syllables (advanced-level). A total of 72 sentences were developed, evenly distributed across the three lexico-grammatical complexity levels at the two lengths bands. Five Chinese language teachers checked the sentences and ensured their naturalness. Following the common practice of administering EI as a proficiency test (Ortega et al., 2002), a 2-s silence was inserted between the end of each sentence and a 0.5-s ringtone prompting the start of the repetition. The EI test went through an iterative process of development and validation and demonstrated good psychometric qualities (Yan et al., 2020).¹ Sample EI sentences of each level are provided below.

这件衣服非常便宜 (Beginning-level, 8 syllables).

This piece of clothing is very cheap.

我每天都坐剬共汽车去学校  (Beginning-level, 12 syllables).

I take the bus to school every day.

他比以前进步多了 (Intermediate-level, 8 syllables).

He made much more progress than before.

他们的房间里挂着一张地图  (Intermediate-level, 12 syllables).

There is a map hanging in their room.

他从事于研究工作 (Advanced-level, 8 syllables).

He does research for his job.

他不但不高兴，反而有点生气  (Advanced-level, 12 syllables).

He is not happy; on the contrary, he is a little angry.

A questionnaire was used in this study to elicit processes and strategies that participants employed for the completion of the EI test. As this is an exploratory study, open-ended questions were designed to elicit participants’ thoughts on the processes and strategies when they were completing the EI test. Although stimulated recall is frequently used for such purposes (Gass and Mackey, 2013), it could not conveniently be implemented when this study was conducted due to the physical and technical constraints posed by the COVID-19 pandemic. Cohen (2014) suggested that questionnaires can also gain insights into test takers’ strategy use when the questionnaires include information about test takers’ thoughts about their behaviors on test tasks. The main questions probing into test-taking strategies in the questionnaire were:

The language background questionnaire gathered basic information about participant demographics, language background, and language learning experience. All the tasks were delivered online via the Gorilla Experiment Builder (Anwyl-Irvine et al., 2019) to accommodate the need of online testing due to the COVID-19 pandemic.