Volume 04 Issue 12-2024
318
International Journal of Advance Scientific Research
(ISSN
–
2750-1396)
VOLUME
04
ISSUE
12
Pages:
318-323
OCLC
–
1368736135
A
BSTRACT
This paper explores the application of artificial intelligence (AI) in generating classroom foreign language
tests, aiming to identify the challenges and opportunities encountered during item creation. Employing a
case study, a qualitative research method, this study used data from a single subject to investigate the
process of generating classroom assessment items using an AI system in a real-world setting. The findings
highlighted several opportunities and challenges faced by the participant when utilizing AI. Analysis of the
feedback revealed three major benefits of using AI: practicality, customization, and efficiency. Conversely,
four significant challenges were identified: readability, validity, insufficient data for other languages, and
issues of ownership.
K
EYWORDS
Artificial intelligence, assessment, exam questions, foreign language teaching and readability.
Journal
Website:
http://sciencebring.co
m/index.php/ijasr
Copyright:
Original
content from this work
may be used under the
terms of the creative
commons
attributes
4.0 licence.
Research Article
DEVELOPMENT OF THE METHODS OF CREATING LANGUAGE
EXAM QUESTIONS USING ARTIFICIAL INTELLIGENCE: A CASE
STUDY
Submission Date:
December 15,
2024,
Accepted Date:
December 20, 2024,
Published Date:
December 30, 2024
Crossref doi:
https://doi.org/10.37547/ijasr-04-12-49
Nodirbek Yuldashev Abdumannob
O’g’l
i
Researcher, Andijan Machine Building Institute, Uzbekistan
Dr. Danish Ather
Instructor, Head Department Of Research, Associate Professor, Amity University In Tashkent, Uzbekistan
Volume 04 Issue 12-2024
319
International Journal of Advance Scientific Research
(ISSN
–
2750-1396)
VOLUME
04
ISSUE
12
Pages:
318-323
OCLC
–
1368736135
I
NTRODUCTION
Assessment is a crucial element in the educational
and business sectors' training and selection
systems. It provides valuable feedback to both
teachers and students regarding the effectiveness
of language instruction, identifies areas of
difficulty, and informs decisions about student
placement in various language courses (Ferrara
et al., 2017; Freddi, 2021). Additionally, it plays a
vital role in the development and refinement of
language curricula. In the context of additional or
foreign language assessment, it helps teachers
gauge
their
learners'
current
language
proficiency and pinpoint areas needing further
instruction,
thereby
identifying
students'
strengths
and
weaknesses
(Brown
&
Abeywickrama, 2010; Purpura, 2016). This
process offers teachers crucial insights into which
language skills their students have mastered and
which areas require more focus to improve their
language abilities. Despite the critical information
provided by language assessments, teachers often
spend considerable time creating assessment
items aligned with the syllabus and content
prescribed by their institutions or their country's
Ministry of Education. These items, which might
include multiple-choice questions, reading
comprehension tasks, or spoken language
activities (Hughes & Hughes, 2020), are labor-
intensive to develop. This involves analyzing the
current and target needs of learners to ensure the
most accurate and efficient evaluation of student
language proficiency. Nonetheless, the time and
effort invested by teachers in this process are
seen as essential for accurately assessing and
supporting teaching and learning, ultimately
contributing to the future success of their
learners.
In my view, the integration of AI in educational
assessment represents a groundbreaking
advancement that has the potential to
revolutionize the way we evaluate and enhance
learning. While the benefits such as increased
efficiency, customization, and practicality are
substantial, it is crucial to address the challenges
head-on to maximize the positive impact of AI.
Issues like readability, validity, data availability
for diverse languages, and ownership must be
thoroughly researched and managed. By doing so,
we can harness the full potential of AI in
education, ensuring that it not only supports
teachers and students but also fosters a more
adaptive and inclusive learning environment.
L
ITERATURE REVIEW
Artificial Intelligence -
‘Artificial Intelligence’ is
described as the "science and engineering of
making
intelligent
machines,
especially
intelligent computer programs" (McCarthy et al.,
2006: 2). AI refers to machine intelligence capable
of performing human-like tasks and activities
(Eaton et al., 2021). Essentially, AI is a platform
designed to think, reason, and act similarly or
even superior to humans. Recent technological
advancements have introduced a wide range of
applications in the education sector, from task
design to automated assessment (Gardner et al.,
Volume 04 Issue 12-2024
320
International Journal of Advance Scientific Research
(ISSN
–
2750-1396)
VOLUME
04
ISSUE
12
Pages:
318-323
OCLC
–
1368736135
2021; Gonzalez-Calatayud et al., 2021; Levy &
Stockwell, 2006; Stephenson & Harvey, 2022).
Students, teachers, and researchers are
increasingly using technological tools to enhance
writing, such as writing assistants (e.g.,
Grammarly), paraphrasing tools (e.g., QuillBot),
research assistants (e.g., Elicit), and reference and
citation checkers (e.g., Reciteworks) (Godwin-
Jones, 2022; Zhang & Zou, 2022). Beyond these
applications, AI is also employed in automated
essay scoring systems and adaptive tests, which
adjust the number and order of questions based
on the test-taker's responses
—
two key
applications of AI in educational assessment
(Gardner et al., 2021). Furthermore, AI can aid
learners and teachers by personalizing
instruction, helping students master content
more quickly and effectively (Baker et al., 2019).
The potential of modern technology in education
is vast, and it will continue to profoundly
influence how we learn and teach (Kilickaya &
Kic-Drgas, 2023).
Language Test Creation using AI
AI can generate automated items without human
intervention, producing a wide range of content,
including blog posts, writing summaries, and
feedback, which has increased the use of AI in
language teaching and learning (UNESCO, 2021;
Yanhua, 2020). This technology enables teachers
to perform more advanced tasks such as
automated grading of essays and oral proficiency
assessments in writing and speaking classes
(Borade & Netak, 2021; Kessler, 2023; Langenfeld
et al., 2022; Yu et al., 2022; Yunjiu et al., 2022). AI
technology extends beyond intelligent language
tutoring and feedback provision to facilitate
interactions between learners and computers in
both text and spoken forms. From my point of
view, the integration of AI into language teaching
and assessment offers tremendous potential to
transform educational practices. The automation
of item creation and assessment tasks not only
enhances efficiency but also allows educators to
focus more on personalized instruction and
meaningful interactions with students. While
there are challenges to address, such as ensuring
the accuracy and fairness of AI-generated
assessments, the benefits far outweigh the
drawbacks. Embracing AI in education can lead to
more effective teaching methods and a deeper
understanding of language use and learning
patterns. As we continue to refine these
technologies, their role in education will
undoubtedly become even more pivotal, offering
unprecedented opportunities for both teachers
and learners.
M
ETHODOLOGY
Research design - The current study is a case
study, a qualitative research method involving an
in-depth, detailed examination of a specific
instance or phenomenon within its real-life
context. This approach aims to investigate a
problem "in situ," within its natural environment,
the language classroom (Creswell, 2007). The
study has an ethnographic character (Jones &
Smith, 2017), utilizing observation as a key tool.
The research aimed to provide a comprehensive
analysis of an individual’s experience, feelings,
and perceptions to explore, analyze, and explain
Volume 04 Issue 12-2024
321
International Journal of Advance Scientific Research
(ISSN
–
2750-1396)
VOLUME
04
ISSUE
12
Pages:
318-323
OCLC
–
1368736135
complex social phenomena. Case studies allow
researchers to gain a thorough understanding of
a single person, group, or event by collecting and
analyzing data from multiple sources. A crucial
aspect of the study design was the meticulous
analysis of collected data and the researcher’s
active involvement in group activities during the
research (Dumont, 2023). By examining a single
case in detail, researchers can uncover patterns,
trends, and insights that may be applicable to
other cases and contexts (Griffee, 2012; Mackey &
Gass, 2022). In this study, the methodology used
is both vital and innovative, as it allows for tracing
a new phenomenon from the teacher’s
perspective, focusing on the authentic use of AI in
education.
Data Collection and Analysis
Data were gathered through a journal maintained
by the author while creating exam questions for
the course. The journal included the following
questions: a) What are the learning objectives of
this course, and how can AI be utilized to create
questions that align with these objectives? b)
How did the use of AI allow you to tailor your
assessment items to the specific needs of your
course and curriculum? c) What opportunities or
benefits did you encounter using AI to create the
questions? d) What are the potential limitations
or challenges of using AI for creating assessment
questions, and how can these be mitigated or
addressed? e) Any other comments or
suggestions?
The participant began documenting responses to
these questions in the journal two weeks before
each assessment period, including the midterm
and final exams. The content analysis method was
applied to analyze the data collected from the
journal (Coffey & Atkinson, 1996). The
researcher’s responses were coded and
categorized based on the journal questions. A
total of 78 entries, comprising 4,006 words, were
analyzed over a four-week period. These
categories were then examined to identify any
patterns or trends. The analysis results were
subsequently used to draw conclusions and make
recommendations.
C
ONCLUSION
Language assessment plays a crucial role in
helping teachers and learners evaluate
educational practices, identify areas of difficulty,
and tailor instruction to meet individual learner
needs. This study aimed to explore the challenges
and opportunities associated with using AI to
create language assessment questions. Conducted
as a case study with a single participant, the
research delved into their experiences and
perceptions. The findings highlighted several
benefits of using AI, such as practicality,
customization, and efficiency in generating
assessment questions. However, challenges such
as readability, validity, lack of data for other
languages, and issues of ownership were also
identified. Despite the advantages, human
intervention remains necessary to ensure the
validity and quality of AI-generated questions.
Limitations of the Study and Suggestions for
Further Research
Volume 04 Issue 12-2024
322
International Journal of Advance Scientific Research
(ISSN
–
2750-1396)
VOLUME
04
ISSUE
12
Pages:
318-323
OCLC
–
1368736135
This study's scope was limited by its single-case
design, which restricts the generalizability of its
findings. Additionally, the potential for
researcher bias exists, as the researcher's
opinions and perspectives could have influenced
the study. The data was confined to the
participant's contributions and available journals,
which may not provide a comprehensive view.
The time frame of the research may have also
limited its ability to capture the complex
dynamics involved in using AI to prepare
questions, including pre-reading questions and
possible answers (Attali et al., 2022; Henrickson,
2021; Killawala et al., 2018; Taylor, 2022). As AI-
generated content becomes more prevalent, from
music to artwork, significant legal questions
about intellectual property rights arise. Future
research should investigate the ownership and
copyright protection (Zurth, 2021) of AI-
generated assessment questions. Additionally,
exploring how teacher revisions to AI-produced
questions affect ownership and other issues, such
as equity (Stephenson & Harvey, 2022) and AI-
based cheating (Fyfe, 2022), will be crucial.
Further studies could also examine the
productive, disruptive, or destructive roles AI
might play in the field of education.
R
EFERENCES
1.
Darvishi, A., Khosravi, H., Sadiq, S., & Gasevic,
D. (2022). Incorporating AI and Learning
Analytics to Build Trustworthy Peer
Assessment Systems. British Journal of
Educational Technology, 53 (4), 844-875.
https://doi.org/10.1111/bjet.13233
2.
Dumont,
G.
(2023).
Immersion
in
Organizational
Ethnography:
Four
Methodological Requirements to Immerse
Oneself in the Field. Organizational Research
Methods,
26
(3),
441-458.
https://doi.org/10.1177/109442812210753
65
3.
Eaton, S. E., Mindzak, M., & Morrison, R. (2021,
May 29 - June 3). Artificial Intelligence,
Algorithmic Writing & Educational Ethics
[Paper Presentation]. Canadian Society for the
Study of Education Societe canadienne pour
l'etude de l'education, Edmonton, AB, Canada.
http://hdl.handle.net/1880/113569
1.
4. Ferrara, S., Lai, E., Reilly, A., & Nichols, P. D.
(2017). Principled Approaches to Assessment
Design, Development, and Implementation in
A. A. Rupp & J. P. Leighton (Eds.), The
Handbook of Cognition and Assessment:
Frameworks,
Methodologies,
and
Applications. 41-72. Hoboken: Wiley
and
Sons.
https://doi.org/10.1002/9781118956588.ch
3
4.
Freddi, M. (2021). Reflection on Digital
Language
Teaching,
Learning,
and
Assessment in Times of Crisis: A View from
Italy. In N. Radic, А. Atabekova, M. Freddi & J.
Schmied (Eds.), The World Universities’
Response to COVID-19: Remote Online
Language Teaching, 279-293. Research-
publishing.net.
https://doi.org/10T4705/rpnet.2021.52.127
8
5.
Fyfe, P. (2022). How to Cheat on Your Final
Paper: Assigning AI for Student Writing. AI &
Volume 04 Issue 12-2024
323
International Journal of Advance Scientific Research
(ISSN
–
2750-1396)
VOLUME
04
ISSUE
12
Pages:
318-323
OCLC
–
1368736135
Society.
https://doi.org/10.1007/s00146-
022-01397-z
6.
Garcia-Penalvo, F. J., Corell, A., Abella-Garcia,
V.,
&
Grande-de-Prado,
M.
(2021).
Recommendations for Mandatory Online
Assessment in Higher Education During the
COVID-19 Pandemic. In D. Burgos, A. Tlili, & A.
Tabacco (Eds.), Radical Solutions for
Education in a Crisis Context: COVID-19 as an
Opportunity for Global Learning (pp. 85-98).
Berlin:
Springer.
https://doi.org/10.1007/978-981-15-7864-
3_7
7.
Gardner, J., O’Leary, M., & Yuan, L. (2021).
Artificial
intelligence
in
educational
assessment: “Breakthrough? or buncombe
and ballyhoo?” Journal of Computer Assisted
Learning,
37(5),
1207-1216.
https://doi.org/10.1111/jcal.12577
8.
Griffee, D. T. (2012). An introduction to
second language research methods: Design
and data. Dale T. Griffee. TESL-EJ Publications.
http://www.tesl-
ej.org/pdf/ej60/sl_research_methods.pdf
9.
Godwin-Jones, R. (2022). Partnering with AI:
Intelligent Writing Assistance and Instructed
Language Learning. Language Learning &
Technology,
26(2),
5-24.
http://doi.org/10125/73474
10.
Mackey, A., & Gass, S. M. (2022). Second
Language Research: Methodology and Design
(3rd ed.). Oxfordshire: Routledge. ISBN
9781032036632
