日本テスト学会誌 Vol.15 No.1 Abstract
トップ>学会誌>既刊号一覧>既刊号（15-1）>Abstract
JART Vol.15 No.1
| ▶ General research | |
|---|---|
| Comparison of Item Selection Algorithms in Computerized Adaptive Testing with a Time Limit | |
| Kyosuke Bunji1,2 | |
| 1Graduate School of Education, The University of Tokyo, 2Japan Society for the Promotion of Science | |
| This study compared several item selection algorithms in a situation where a time limit exists in terms of estimation accuracy and item bank security. Simulation results show the following: (1)adjusting item information using the expected response time will improve both the estimation accuracy and the item bank security; (2)when item discrimination and expected response time are positively correlated, the mean numbers of the answered items decrease particularly in the algorithms which determine the next item according to the information; (3)under the condition in (2)algorithms that suppress the inflation of overlap rate demonstrate better estimation accuracy and; (4)the root-mean-square error and test information show slightly different results between algorithms. | |
| Keywords: Item Response Theory, Adaptive Testing, Item Selection Algorithm, Response Time | |
| ▶ General research | |
| A re-formulation of the DINA model and a direct derivation of estimation algorithms | |
| Kazuhiro Yamaguchi1,2 | |
| 1Hosei University, 2The Japan Society for the Promotion of Science | |
| In this study, we focused on the deterministic input noisy and-gate (DINA) model, which is one of the most fundamental diagnostic classification models, and re-formulated the model to easily derive parameter estimation algorithms. The formulation of this study was characterized by a latent indicator variable to represent individual attribute mastery patterns, which is another form of latent class. The expectation-maximization (EM) algorithm for maximum likelihood (ML) and maximum a posteriori estimation, as well as the Gibbs sampling algorithm, were easily derived because of the re-formulation. In addition, a numerical method to obtain standard errors of maximum likelihood estimation was introduced. A simulation study showed that the re-derived EM algorithm could recover true values, and the standard error estimation method provided appropriate values. The estimation method derived in this study indicated the same estimated values as a previous study method that used real data. | |
| Keywords: diagnostic classification model, cognitive diagnostic model, DINA model, EM algorithm, standard error | |
| ▶ General research | |
| Computer Adaptive Testing that Incorporates Response Time for Efficient Estimation of Non-cognitive Skills | |
| Kazuya Fujita, Kensuke Okada | |
| Graduate School of Education, The University of Tokyo | |
| In the current study, we propose the computer adaptive extension of the item response and response time model for measuring non-cognitive skills. For this purpose, the item information function of the joint item response and response time model that reflects the psychological theories of non-cognitive skill measurement is utilized. Based both on the respondents’ item responses and response time information, the next item to be presented is chosen so as to maximize the item information per unit time. Simulation study and empirical application confirmed that the proposed computer adaptive testing with response time information result in higher estimation precision than selecting items at random. This means that the proposed method would be effective in reducing the number of items required on the test. Also, simulation revealed that when items in the item bank have a diverse range of parameter values with regard to response time, the maximum item information per unit time criterion is much better than in traditional approaches. We conclude that computer adaptive presentation of items based on the proposed method would be an effective way of estimating non-cognitive skills. | |
| Keywords: computerized adaptive test, response time, item information per time unit, non-cognitive skills | |
| ▶ Case study | |
| Evaluation of Summarizing Patterns When Reading English Passages: Based on Educational Measurement | |
| Takahiro Terao1,2, Hidetoki Ishii1 | |
| 1Nagoya University, 2Japan Society for the Promotion of Science | |
| The purpose of this study is to examine summarizing patterns in English reading and summarizing test items by proficiency level. Along with the summarization model proposed in the area of cognitive psychology, we developed four types of summarizing test items: 1) deleting unnecessary information from a summary, 2) generalizing a list of concrete examples, 3) integrating the entire information in the paragraph, and 4) summarizing the whole passage. Test takers were 60 undergraduate students, and were asked to answer a set of these reading and summarizing test items. After that, we asked three collaborators to classify students’ summaries into given categories. Tracelines showed that less proficient test takers included unnecessary information or concrete descriptions in their summaries, while middle-proficient students contained a biased statement or a different one from that of the writer. The finding in the current study may contribute to the development of test items or scoring rubrics. | |
| Keywords: constructed-response reading and summarizing test items, summarizing patterns, typical errors, trace lines | |
| ▶ Case study | |
| Trends of the applicants who used the National Center Test scores to apply excessively numerous numbers of private universities; yearly shifts, local characteristics, social factors | |
| Teruhisa Uchida1, Taka-Mitsu Hashimoto2 | |
| 1The National Center for University Entrance Examinations, 2Teikyo University | |
| We analyzed yearly trends among the applicants who applied excessively numerous private universities using their scores of the National Center Test. The results showed three distinctive periods. In the first period from 2008 to 2011, these kinds of applicants were scattered in many different areas around the country. In the second period from 2012 to 2015, such applicants rapidly increased in the areas hit by the 2011 East Japan Earthquake and Tsunami, then decreased in three years. This trend was caused by the fee exemptions by universities to support the young people in the area. During the third period from 2016 to 2017, the numbers of applicants who applied numerous private universities first increased in the metropolitan area. Then this trend was expanded to other areas. The factors affected this trend were a) discounts on admission fees when applicants applied multiple departments, and b) simplified procedures by web-based application systems. Meanwhile, the actual numbers to apply private universities using the National Center Test scores increased nationally, but decreased in the prefectures which were less populated. In other words, the trends were opposite between populated and depopulated areas. | |
| Keywords: the National Center Test, population of 18-year-olds, the Great East Japan Earthquake, the Great Kumamoto Earthquake, Admission Fee Exemptions | |
| ▶ Case study | |
| High school opinion on university admission confronting articulation reform policy between high schools and universities: The case of Admissions Office Entrance Examinations at Tohoku University | |
| Naoki T. Kuramoto, Tomohiro Miyamoto, Hiroyuki Nagahama | |
| Tohoku University | |
| A drastic system reform is planned for university entrance examinations in FY 2021. The changes that are expected to have the most influence are the use of English language tests provided by private companies and the introduction of constructed-response questions in joint examinations. There are also changes that will affect specific universities differently. Tohoku University was forced to revise its AO type III examination, as the schedule for reporting joint examination scores to universities will be delayed by the introduction of the constructed-response questions. In order to reflect high school opinion in revisions of university admission policy, we delivered a questionnaire to high schools which produce large numbers of enrolled students at our University. The items included questions about introducing a self-scoring method to the first-stage selection of AO type III and using constructed-response questions so on. In the results, approval for a self-scoring system was about 60%. In contrast, there was strong opposition to the use of construct-response questions. Tohoku University released a preliminary announcement concerning the FY2021 entrance examination based on the survey results. | |
| Keywords: articulation reform policy between high schools and universities, joint university examinations, self-scoring, constructed responses | |
| ▶ Case study | |
| Reanalysis of Local Authority Achievement Test with IRT and LCGA Trends of Mathematics Achievement Score Gap | |
| Toshiaki Kawaguchi1, Go Matsuo1, Toshiaki Isobe2, Yusuke Higuchi1 | |
| 1University of Teacher Education Fukuoka, 2Chikushino City Board of Education | |
| In this paper, we apply "Equating" by Item Response Theory (IRT) to some mathematics achievement surveys carried out by A City educational administration. Besides, to analyze with Latent Class Growth Analysis (LCGA), we investigate the trends of achievement gap from fourth to ninth grade. Findings are as follows. First, as a result of the analysis with LCGA, the changes of academic ability from fourth to ninth grade are divided into four groups. The differences in educational achievement among these four groups have already appeared at the time of fourth grade, and the trajectory of one group does not intersect with that of another group. Second, the characteristics of children differ among the four groups. The lower the academic ability is, the higher the proportion of those receiving schooling aid is. Also, our analysis suggests that boys' academic ability tend to be polarized. Without IRT, we cannot found growing achievement gaps, it is possible that analysis without IRT underestimates inequality. | |
| Keywords: Item Response Theory, Local Authority Achievement Test, Equating, Achievement Gap, Mathematics | |
| ▶ Case study | |
| Causal Effect of Inquiry Teaching on Science Achievement in PISA 2015 in Japan | |
| Kan Hishiyama, Kensuke Okada | |
| Graduate School of Education, the University of Tokyo | |
| Inquiry teaching, which emphasizes the autonomy of students in learning, has become popular in science education because it is considered to promote students’ learning. However, previous studies have found mixed evidence regarding the effect of inquiry teaching on achievement; some found positive effects but others found the opposite. Hence, this paper investigates the causal effect of inquiry teaching on science achievement in Japan using the data from the Programme for International Student Assessment (PISA) 2015. We first classified the students into five groups based on the level of inquiry teaching that the students received. Using this as a treatment variable, we estimated the generalized propensity score by multinomial logistic regression with 38 covariates. Further, we estimated the causal effect of the treatment on science achievement using the inverse probability weighting estimator. Achievement was found to be the highest in the group that received moderate inquiry teaching, and decreased in groups that received more excessive levels. This result is in line with the previous study that used former PISA data. We thereby conclude that inquiry teaching has a positive causal effect on achievement when conducted at a moderate level; however, its excessive use can result in a negative causal effect. | |
| Keywords: inquiry teaching, statistical causal inference, generalized propensity score, IPW estimator | |
| ▶ Case study | |
| Clarifying Solving Processes of Fill-in-the-Blank Problems Using the Think-aloud Method ― Using Past Mathematics Tests of the National Center University Entrance Examination ― | |
| Miho Kawasaki1, Hajime Shirouzu2, Hiroyuki Masukawa3 | |
| 1Shizuoka University, 2University of Tokyo, 3University of the Sacred Heart, Tokyo | |
| This study examines the processes of thinking about and solving fill-in-the-blank problems using the think-aloud method. We asked 17 high school students to solve two problems in the subject of mathematics, which were questions that had been set in past tests of the National Center for University Entrance Examinations. Here, we found 14 students who solved the problems in their numerical order. Only two students noticed an efficient way of solving the problems based on their insight into the relationship between the items in the problems. Thus, few students were aware of how each problem was composed of a number of items or how the items related to each other. Five students tried to adjust their answers to fit the blank, the two out of which changed their answer with no justification. This was the case even with high scorers. These results suggest the following features of the mathematics tests of the National Center University Entrance Examinations: (1) the path of the items mapped out by the questioner led the student to show appropriate but fragmented knowledge and skills with little insight into the problem, and (2) time pressure made students skimp on efforts to follow the path as led by the items to seek solutions taking in the entire problem. The findings contribute to the discussion about the purpose of the examinations and problem solving processes induced by them. | |
| Keywords: cognitive lab, think-aloud method, problem solving process, test format, National Centre University Entrance Examination，mathematics | |