Volume 03 Issue 10-2023
65
American Journal Of Philological Sciences
(ISSN
–
2771-2273)
VOLUME
03
ISSUE
10
P
AGES
:
65-70
SJIF
I
MPACT
FACTOR
(2022:
5.
445
)
(2023:
6.
555
)
OCLC
–
1121105677
Publisher:
Oscar Publishing Services
Servi
ABSTRACT
The purpose of this study was to evaluate whether the analytic reporting approach would have a greater impact on
improving English language learners' verbal skills. The two groups, the analytic group, and the holistic group, were
compared in terms of the test results from the analytic scoring approach and holistic scoring approach respectively.
Furthermore, the study revealed that students were more than willing to adopt an analytical scoring method for
teaching and assessment of speaking skills.
KEYWORDS
Speaking, analytic scoring approach, holistic scoring approach, EFL.
INTRODUCTION
At the University for Natural Resources and
Environment in Ho Chi Minh City, a holistic approach to
ratings has given rise to teaching and assessing the
speaking skills of EFL students. However, useful
anatomical guidance for students to improve their
speaking skills is not provided by the holistic grading
approach in teaching and assessment. Obtaining the
score for speaking, students do not know why they
succeed or fail since a single score does not allow
[students] to distinguish between various aspects of
speaking such as control of syntax, depth of
vocabulary, organization, and so on.
Research Article
TEACHING AND ASSESSING SPEAKING PERFORMANCE THROUGH
ANALYTIC SCORING APPROACH
Submission Date:
October 08, 2023,
Accepted Date:
October 13, 2023,
Published Date:
October 18, 2023
Crossref doi:
https://doi.org/10.37547/ajps/Volume03Issue10-09
Turobova Malika Baxrom Qizi
Doctoral Student Of Uzbekistan State World Languages University, Uzbekistan
Journal
Website:
https://theusajournals.
com/index.php/ajps
Copyright:
Original
content from this work
may be used under the
terms of the creative
commons
attributes
4.0 licence.
Volume 03 Issue 10-2023
66
American Journal Of Philological Sciences
(ISSN
–
2771-2273)
VOLUME
03
ISSUE
10
P
AGES
:
65-70
SJIF
I
MPACT
FACTOR
(2022:
5.
445
)
(2023:
6.
555
)
OCLC
–
1121105677
Publisher:
Oscar Publishing Services
Servi
A framework to test based on effectiveness was
proposed by Bachman and Palmer in 1996. To help
teachers, choose the type of test that they use, this
framework can play a role. This framework proposed
six qualities of test usefulness: Reliability, Construct
Validity, Authenticity, Interactiveness, Impact, and
Practicality. Weigle commented in 2002 on the
framework of Bachman and Palmer, showing that
these same six criteria for test effectiveness are
combined to form a holistic and analytical scale.
Analytical scales are more reliable in terms of reliability,
even though the holistic scoring scale is acceptable.
Analytical scales are more appropriate for Second
Language students when it comes to the validity of a
construction. In terms of impact, the analytic scale will
provide students with greater information on their
abilities; teachers and trainers can also benefit from it
because it allows them to determine which teaching
methods are appropriate for each student. However,
the analytic scales will be more disadvantageous than
the holistic scales when practicality is taken into
consideration since it is a time-consuming and
expensive method of scoring.
This research aimed to examine if the analytic scoring
approach in teaching and assessment would be more
effective
for
improving
students’
speaking
performance. The student's attitude to the
introduction of an analytic scoring method in speaking
classes and assessments was also investigated. The
research question to be addressed in the course of
these studies is thus:
1. Is the ability of EFL students to communicate in a
coherent, holistic way more improved than they are
taught and assessed by using language skills
analytically?
2. How do EFL students reflect on the implementation
of the analytic scoring approach in speaking teaching
and assessment?
Analytic Scoring Schemes
Analytic scoring schemes are methods of evaluation
that divide the goal of finished items into criterion
pieces, with each part being assessed separately. This
method's methods entails categorizing a discourse's
many aspects for the goal of scoring. The sum of the
ratings for each component under consideration
makes up the final score. To prevent bias against the
entire product when employing analytical scoring
schemes, it is vital to treat each criterion or component
as a separate entity. Speaking performance may be
graded on factors including content, organization,
cohesiveness, register, vocabulary, grammar, or
mechanics depending on the assessment's goals. This
way of scoring, in contrast to the holistic scoring
scheme, prevents the possible weakness in global
impression band scales of unequal development in the
many criteria. Additionally, with this method of
evaluation, a teacher can easily assign a higher score
Volume 03 Issue 10-2023
67
American Journal Of Philological Sciences
(ISSN
–
2771-2273)
VOLUME
03
ISSUE
10
P
AGES
:
65-70
SJIF
I
MPACT
FACTOR
(2022:
5.
445
)
(2023:
6.
555
)
OCLC
–
1121105677
Publisher:
Oscar Publishing Services
Servi
for a particular criterion by designating a specific
coefficient when they believe that their pupils should
pay close attention to the criterion. For instance, if a
teacher places more emphasis on how the speech is
organized, he or she may assign the criteria a
coefficient of two before calculating the overall grade
for the speaking performance.
Advantages of Analytic Scoring Schemes
Many speaking professionals prefer analytical scoring
schemes over holistic ones for a variety of reasons. As
was already indicated, it first offers more helpful
diagnostic data concerning pupils' speaking ability. In
other words, it identifies a learner's areas of strength
and weakness. Because it accesses the examinee’s
strengths and weaknesses and pinpoints the precise
elements of speaking discourse that an examinee
needs to develop, analytical scoring has been seen as a
more interpretable scoring approach. Although
analytical ratings offer more diagnostic information,
holistic scores are still important for classifying
speaking ability in its entirety. Additionally, the data
enables teachers and curriculum designers to better
adapt their lessons to the needs of their students. The
explicitness of the analytic scoring scheme guides
offers teachers a potentially useful tool for giving
speakers consistent and direct feedback, according to
Park. Second, analytical scoring schemes are especially
helpful for second language learners who are more
likely to exhibit a distinct or uneven profile across
various spoken discourse elements. Others may have
strong control of sentence structure but struggle with
logical speech arrangement. Some second language
learners may have outstanding speaking skills in terms
of organization and content but may have considerably
weaker grammatical control. When the same rubric
categories are used frequently, analytic scoring scales
can demonstrate to students their advancement over
time in some or all dimensions.
Disadvantages of Analytic Scoring Schemes
It takes a long time to rate speaking ability as the
examiners have to make more than one decision on
every performance, so it is very difficult to score
analytically. When scoring analytically, an examiner has
to check, consider, and score each criterion of the
speaking ability and then give a total score depending
on the coefficient put forward.
Critics of analytic scoring schemes also point out that
measuring the quality of a text by tallying accumulated
sub-skill scores diminishes the interconnectedness of
spoken discourse. The whole should be larger than the
sum of its parts at that point, it is thought. Measuring
the quality of a spoken discourse by tallying
accumulated sub-skills gives the false impression that
speaking can be understood and fairly assessed by
analyzing autonomous discourse features. Hughes
found that focusing on individual aspects may distract
attention from the general impact of a speech. A
Volume 03 Issue 10-2023
68
American Journal Of Philological Sciences
(ISSN
–
2771-2273)
VOLUME
03
ISSUE
10
P
AGES
:
65-70
SJIF
I
MPACT
FACTOR
(2022:
5.
445
)
(2023:
6.
555
)
OCLC
–
1121105677
Publisher:
Oscar Publishing Services
Servi
composite score may be highly reliable but not valid, as
it frequently exceeds the sum of several components.
In this sense, analytic score tends to decrease and
overcomplicate the elements of speech as well as
emphasize deficiencies in comparison with their
strengths.
Hughes, warned that in scoring analytically, the
criterion scored first may affect subsequent criteria
which are scored later, making the overall effect of a
speech diverted to an individual criterion. Fulcher,
basing his idea on Thorndike’s Thought, describes this
phenomenon as a halo effect in analytical scoring.
Even experienced essay judges can struggle to give
numerical values based on specific descriptors, which
is another issue with some analytic grading techniques.
Scorers can disagree with one another in this area. In
comparison to a single score produced by a holistic
scale, it is more challenging to achieve intra- and inter-
rater dependability on all of the characteristics in an
analytical scoring scheme. Additionally, McNamara
revealed evidence demonstrating that scorers tend to
evaluate grammar-related categories more harshly
than they do other categories, overemphasizing the
importance of accuracy in constructing a profile of
students' proficiency. This drawback is unavoidable,
especially when using inexperienced or untrained
scorers. Grammar-related categories are somewhat
wrong
–
right categories whereas other categories are
judgments.
METHODOLOGY
Students
The two classes at the University for Natural Resources
and Environment in Ho Chi Minh City, totaling 104
students, were invited to take part in the study as the
experimental group (the analytical group), which used
the analytic scoring approach to teach and evaluate
speaking, and the control group (the holistic group),
which used the holistic scoring approach.
51 students made up the analytic group, of which 13
were male and 38 were female, making up 74.51% and
25.49%, respectively. This class was selected as the
analytic group in this study because its mean on the
pretest was 6.78, lower than the holistic group's mean
of 6.81. When the holistic group first appeared to
perform better than the analytical group, this decision
would make the study more reliable.
53 students made up the holistic group; 40 of them
were female, making up 75.47% of the group, and 13
were male, making up 24.53%.
Teacher
To reduce the researcher's prejudice, a female teacher
was asked to instruct the two groups. The instructor
has been teaching speaking for more than six years and
has a master's degree in TESOL.
Pretest and posttest
Volume 03 Issue 10-2023
69
American Journal Of Philological Sciences
(ISSN
–
2771-2273)
VOLUME
03
ISSUE
10
P
AGES
:
65-70
SJIF
I
MPACT
FACTOR
(2022:
5.
445
)
(2023:
6.
555
)
OCLC
–
1121105677
Publisher:
Oscar Publishing Services
Servi
The pretest was the last exam the two groups of
students took for the speaking course in their previous
semester. Six speaking tests from the speaking
course's posttest were combined to create the
posttest's final score.
The measures recommended by Nakamura and
Hughey served as the basis for the analytical scoring
scale used in this work. Each of the five Nakamura
criteria originality of content, organization, vocabulary,
grammar, and logical consistency is scored on a scale
of four. Five criteria make up the Hughey et al. scale:
content, organization, vocabulary, language use, and
mechanics. Five factors make up the analytical scoring
scale that served as the foundation for this study's
instruction and evaluation of speaking: 1) Coherence,
2) Content, 3) Grammar and Structure, 4) Language
used (consisting of Vocabulary, Spelling, Word used),
and 5) Organization.
Questionnaire
The study's questionnaire was a six-scale Likert-type
survey in Vietnamese. By making the most neutral
response possible, the six-scale response was utilized
in the questionnaire to stop respondents from
selecting a "sit the fence" attitude. The questionnaire
was created to allow respondents to indicate whether
they have a favorable or unfavorable opinion of the
employment of the analytical scoring approach.
The study provides insight into how the analytical
method and students' progress in speaking instruction
are related. This understanding also enables teachers
to make an informed choice on the use of the analytical
scoring system to evaluate the speaking abilities of
their students. The holistic scoring system may have
certain benefits for teachers when instructing and
grading oral presentations, but it may have drawbacks
for students who are learning speaking skills on their
own.
REFERENCES
1.
Bachman, L. F. & Palmer, A.S. (1996). Language
Testing In Practice (second edition). Oxford:
Oxford University Press.
2.
Weigle, S.S. (2002). Assessing Writing.
Cambridge, UK: Cambridge University Press.
3.
Park, T. (2004). An Overview of Portfolio-based
Writing Assessment. Retrieved on July 14 2011
from
http://www.tc.columbia.edu/academic/tesol/
WJFiles/pdf/TaejoonParkForum.pdf.
4.
Weir, C.J. (2005). Language Testing and
Validation: An Evidence-based Approach.
Hound grave, Hampshire: Palgrave MacMillan.
5.
Downing, S.M. & Haladyna, T.M. (2006).
Handbook of Test Development. New Jersey:
Lawrence Erlbaum Associates, Inc.
6.
Park, T. (2004). An Overview of Portfolio-based
Writing Assessment. Retrieved on July 14 2011
Volume 03 Issue 10-2023
70
American Journal Of Philological Sciences
(ISSN
–
2771-2273)
VOLUME
03
ISSUE
10
P
AGES
:
65-70
SJIF
I
MPACT
FACTOR
(2022:
5.
445
)
(2023:
6.
555
)
OCLC
–
1121105677
Publisher:
Oscar Publishing Services
Servi
from
http://www.tc.columbia.edu/academic/tesol/
WJFiles/pdf/TaejoonParkForum.pdf.
7.
Moskal, B.M. (2000). "Scoring rubrics: What,
when and how?" Practical Assessment,
Research & Evaluation, 7 (3). Retried on July 11
2011
from
http://pareonline.net/getvn.asp?v=7&n=3.
8.
Park, T. (2004). An Overview of Portfolio-based
Writing Assessment. Retrieved on July 14 2011
from
http://www.tc.columbia.edu/academic/tesol/
WJFiles/pdf/TaejoonParkForum.pdf.
9.
Hughes, A. (1989/2003). Testing for Language
Teachers. Cambridge: Cambridge University
Press.
10.
Hughes, A. (1989/2003). Testing for Language
Teachers. Cambridge: Cambridge University
Press.
11.
Fulcher, G. (2009). Rating Scales and the Halo
Effect. Retrieved on July 17 2011 from
http://languagetesting.info/gf/glennfulcher.ph
p.
12.
Hamp-Lyons, L. (1989). Second language
writing: Assessment issues. In Kroll, B. (Ed.),
Second
language
writing (pp.
69-87).
Cambridge: Cambridge University Press.
13.
McNamara, T. (1996). Measuring second
language performance. London: Longman.
14.
Nakamura, Y. (2004). A Rasch-based analysis of
an in-house English placement test. Retrieved
on
July
2
2011
from:
http://jalt.org/pansig/2004/HTML/Nakamura.ht
m.
15.
Hughey, J.B., Wormuth, D.R., Hartfiel, V.F. &
Jacobs, H.L. (1983). Teaching ESL composition:
Principles and techniques. Rowley, MA:
Newbury House.
