‘jacqueline-pace’, ‘louisella-bonello’

---

© Publications Committee, Faculty of Education, 2007 University of Malta, Msida, Malta MSD06 ISSN 1726-9725

Jacqueline Pace

jacqueline.pace@um.edu.mt

Jacqueline Pace graduated in Education in 1991 specialising in Biology and Physics. For the

last twelve years she has lectured in Physics at the University Junior College. Before, she

taught Physics at the State Sixth Form as well as Physics and Biology at Area Secondary

Schools and Junior Lyceums. She is currently involved in studying certification issues in the

context of low-achieving students.

Louisella Bonello

louisella.bonello@um.edu.mt

Louisella Bonello is a graduate of the Faculty of Education, University of Malta, and holds

an Honours degree in Education, specializing in Physics. After a short stint teaching Physics

at Secondary level, Louisella moved to the G.F. Abela Sixth Form, where she taught Physics

at Advanced level. Louisella has been an examiner of the Ordinary Level Physics Matric

Exam for the past seven years. She attends annual conferences appositely designed for

teachers of Physics as well as others targeting researchers in Physics. She currently holds the

post of Asst Lecturer in Physics at the University Junior College, where she has been

lecturing since the College’s inception.

Abstract:

Predicting the future performance of a student based on the past performance is no

easy task. Developing such a tool would enable Colleges and Universities as well

as other institutions to spend their resources more wisely. This study is an attempt

to use the Physics SEC result as a predictor for performance at Physics Matric

Advanced level. Furthermore, five college-based assessment results taken during

the students’ two year course are also employed as predictors. The study shows

that the number of females pursuing the study of Physics at Advanced level is rather

low when compared to the males. More significantly, all the assessment tools

examined are modest predictors of performance. The Physics SEC result does give

a rough indication of Matric performance. However, achieving a bad grade at SEC

level is not a barrier towards achieving a good result at Matric level. This means

that making the Physics Matric course more stringent is not the way to making it

better.

Vol:4 No.2 2006 37 53

http://www.educ.um.edu.mt/jmer

Predictors of Overall Performance in

Physics Matric Advanced Level:

An insight into entry requirements

Introduction

The main thrust behind this short study is to determine the best predictors of

performance and success in Physics Advanced level, if there are any. This is very

useful information to any educational institution because once a valid set of predictors

is identified these may be employed for a variety of uses, depending on how

sophisticated they are. First of all, predictors may be used to direct students in the

choice of subjects that they undertake for their studies. They may be used to identify

students who may have problems in making the grade early on such that remedial

action may be taken. Good predictive criteria may also be used to point out the

‘gifted’ such that they may be provided with challenges to their learning. The

analysis of students’ performance over a period of time may also be a good

performance indicator of the educational institution, in our case the Junior College.

Finally, the main aim for such an analysis was driven by a belief that the present entry

requirements at the Junior College are not providing proper guidance to the entrants in

their choice of subjects.

1. Setting

At the end of secondary schooling students sit for external examinations that, apart

from accrediting the students, provide access to post-secondary schooling. One of

Malta’s post-secondary academic institutions is the University Junior College. It was

set up in 1995 and caters for those students aiming at tertiary education within the

University itself (Junior College, 2000a).

At Secondary level, students have to make choices of curriculum subjects according

to their abilities, skills and career orientation. They may seek vocational advice from

their subject teachers and, particularly, from school guidance teachers (Education

Division, 2000). As they start their fifth year, secondary students have to take

decisions that are mostly related to the external examinations that they intend to sit for

at the very end of their compulsory schooling. A variety of examination-related

decisions have to be made, such as: which examinations are required to further their

education; what special requirements are needed to follow particular post-secondary

courses; which SEC (Secondary Education Certificate) examination differentiated

paper, A or B, should be attempted (Pace, 2002b).

The entry requirements at the Junior College have ever since its inception been six

passes in the Secondary Education Certificate (SEC) Examination at least at Grade 5

or equivalent (Junior College, 2000b). Around 90% of Junior College entrants

present Physics as their Science requirement (Pace, 2002b). Junior College study

course is based on two academic years during which the students follow 2 A Levels, 3

Intermediates and Systems of Knowledge (Junior College, 2000b).

In the context of mass higher education, the number of students opting to continue

their post-secondary studies is always on the increase. The most challenging aspect

of mass education is that the College had to remodel itself from one catering for the

country’s narrow top ability students to a more comprehensive one which

accommodates students with differing abilities. This brings about additional problems

especially as regards choice of subjects. While a ‘gifted’ student is generally able to

deal with a wrong choice of subject and maybe redirect him/herself in good time, an

average candidate may find that making a wrong choice leads inevitably to a complete

failure in higher education. Proper guidance needs to be provided both at the feeder

schools and during the critical phase of the summer months prior to entry at the Junior

College so that the students may make well-informed choices and also during their

first year if any change of subjects need to be made.

2. Aims

Casual observations of students’ performance in successive year groups suggested

that a good number of Physics students were not doing as well as one would desire.

This trend is immediately uncovered by the low performance in informal tests that the

Physics lecturers set their students and by the considerable proportion of first year

students that have to sit for the Resit due to failure in obtaining a score of more than

45%, in the formal end-of-first-year test.

These insights, together with other research findings, have helped this study in

clarifying the broad issue of performance in Physics Matric Level by addressing it

through more specific research questions.

1. How valid is the Physics SEC examination as a predictor of performance

in the Junior College Advanced Physics Course and in Matric (Advanced)

examination?

2. How valid is the performance in both formal and informal assessments as a

predictor of overall performance in Matric?

3. Are there any significant differences in performance related to gender and

to the SEC differentiated paper?

4. In the light of the evidence from research questions 1 to 3, should the entry

requirements for Physics A level course be more stringent?

3. Methodology and Sample

In order to seek answers for the research questions the College Physics data were

coupled with the MATSEC (Matriculation and Secondary Examinations) data. The

merging of the data and the relevant statistical analyses were carried out using SPSS.

The sample consisted of the SEC 1999 and 2001 fresher students who finished their

studies at the Junior College and sat for the 2000 and 2002 Advanced level Matric

session respectively. This sample also included a number of students who had started

the course in the year 1999 and 1998, and had repeated the first year. The sample of

students who have completed their studies in 2000 will be referred to as the 2000

cohort; while those that completed their studies in 2002 will be referred to as the 2002

cohort. Various filters were used in SPSS to ensure that the data was compared on a

like-with-like basis.

Table 1 shows that there are more boys than girls undertaking the papers and

assessments in Physics. For the 2000 cohort the distribution by paper choice was not

available, although informal analysis yielded a similar balance. The 2002 cohort

distribution shows that there is a larger proportion of students who had chosen paper

A rather than paper-B. The very low number of female, paper-B students must

however stand out as they constitute only about 12% as opposed to the more than 20%

of males.

Table 1. Matric Sample Distribution by gender and SEC paper choice

Gender Male Female Total 2000 Cohort SEC Paper 1998 (sample by paper choice not available)

96 63 159

A 83 43 126

SEC Paper 1999^ 2002 Cohort B^^21 6 27

Total 104 49 153

Total 200 112 312

Besides using the MATSEC data for Matric and SEC, other Junior College internal

data was used. The students’ five assessment marks were entered in the data. These

are divided into two: the first three Assessments take place in the First Year, while the

last two are for the Second Year. These assessment exercises are all lecturer-based

and no attempt at harmonisation is actually in place. The lecturers thus use various

assessment tools such as homework, tests, projects and assignments. The data was

also supplemented by the end-of-first-year test which is a very important test for the

Junior College in that it influences the promotion of students to the Second Year.

There are many different analyses that have been carried out and are being presented

here. All the assessment tools were correlated against each other to find out degrees

of commonality using Pearson product-moment correlation index. The data was also

analysed for gender and SEC paper choice differences using Independent Samples t

test and chi-square tests. Furthermore, the data was recoded into two new groupings,

mainly SEC ability groups and Matric ability groups. For the SEC ability groups, the

data was divided into the top ability (grades 1, 2 and 3) and the bottom ability (4 and

5) (sic). Similarly, the Matric ability group consisted of top ability (grades A, B and

C) and bottom ability (D, E and F) candidates (sic). These groupings permit better

contrasting of data such that differences in performance are sharpened and so

conclusions assume higher validity. Each of the statistical tests used, as mentioned

below, were set to test the null hypothesis at 2-tail significance so as to ensure a more

rigorous statistic.

4. Results

In this part of the study the data analysed is presented systematically using tables and

appropriate figures to outline the main findings. The data is organised into different

groupings, namely according to the differentiated paper chosen at SEC level and by

gender. Furthermore, ability groups have also been created to provide better contrast

between high and low achievers; both for SEC grades as well as for Matric grades.

5.1 Relationship between Different Assessment and Performance

Components

Table 2 shows that the assessments that students have been subjected to as from their

SEC examination (1999, 2000) to their Matriculation in Physics (2000, 2002) do not

show great similarity. Although all the scores, except one, show significant positive

correlation amongst themselves, these differences are not so marked.

Table 2. Product–Moment Correlation Coefficients between Assessment Scores and Examination Grades for A level students 1st Year Assessments1st Year Assessments1st Year Assessments1st Year Assessments 2222

ndndndnd (^) YearYearYearYear AAAAssessmentsssessmentsssessmentsssessments 2000 Cohort2000 Cohort 2000 Cohort2000 Cohort Grades andGrades andGrades andGrades and ScoresScoresScoresScores SECSECSECSEC SECSECSECSEC 1.000 1111 1111 0.472 1.000 2222 2222 0.343 0.508 1.000 3333 3333 0.370 0.552 0.671 1.000 End ofEnd ofEnd ofEnd of First YearFirst YearFirst YearFirst Year TestTestTestTest 1111 stststst YearYearYearYear AssmtAssmtAssmtAssmt TestTestTestTest 0.540 0.459 0.404 0.564 1.000 4444 4444 0.364 0.439 0.181* 0.443 0.544 1.000 5555 2222 ndndndnd YearYear YearYear AssmtAssmtAssmtAssmt^5555 0.420^ 0.436^ 0.413^ 0.555^ 0.606^ 0.583^ 1.000 MatricMatricMatricMatric A levelA levelA levelA level 0.488^ 0.443^ 0.378^ 0.497^ 0.708^ 0.545^ 0.630 1st Year Assessments1st Year Assessments1st Year Assessments1st Year Assessments 2222 ndndndnd (^) YearYearYearYear AssessmentsAssessmentsAssessmentsAssessments 2002 Cohort2002 Cohort 2002 Cohort2002 Cohort Grades andGrades andGrades andGrades and ScoresScoresScoresScores SECSECSECSEC SECSECSECSEC 1.000 1111 1111 0.416 1.000 2222 2222 0.244 0.627 1.000 3333 3333 0.151NS^ 0.498 0.634 1.000 End ofEnd ofEnd ofEnd of First YearFirst YearFirst YearFirst Year TestTestTestTest 1111 stststst YearYearYearYear AssmtAssmtAssmtAssmt TestTestTestTest 0.422 0.315 0.348 0.299 1.000 4444 4444 0.354 0.349 0.306 0.272 0.382 1.000 5555 2222 ndndndnd YearYear YearYear AssmtAssmtAssmtAssmt^5555 0.296^ 0.423^ 0.388^ 0.382^ 0.372^ 0.574^ 1.000 MatricMatricMatricMatric A levelA levelA levelA level 0.442^ 0.463^ 0.393^ 0.318^ 0.550^ 0.415^ 0.617 One r-value marked NS^ is not significant. Another one marked * is significant to the 0.05 level. All other r-values are significant at the 0.01 level

In general, the correlation values, albeit significant, are all rather low values. Few of

these assessment tools correlate well with each other. The SEC exam correlates more

with other summative-type assessments namely the Matric and the End of First Year

test than it does with any of the five informal and formative assessments.

The end-of-first-year test shows an incredible metamorphosis. While in the 2000

cohort it correlates rather positively with all the other assessments, in the 2002 cohort

it correlates appreciably well only with SEC and Matric A level but rather poorly with

the other assessment exercises. This is largely due to a change in the system used for

test construction. While in the latter only one person was responsible for the test

construction, in the former the group effort used yielded a better paper.

The other assessment tools seem more or less to correlate well with each other.

Although there are some peculiar differences, in that the First Year Assessments

correlate very well between themselves and not so much with the 2 nd^ Year

Assessments. On the other hand the 2nd^ Year assessments compare well and are

generally a good predictor of the Matric result, especially Assessment 5, the last one.

5.2 Gender issues

The most striking difference between the students is and remains the heavy deficit of

girls choosing to study Physics at Advanced level, where they are outnumbered by a

ratio higher than 2:1. On the other hand, there are very little differences between male

and female students throughout the two years of study.

A look at the t-values in Table 3 reveals that there are little to no gender differences

within this group of students. Pearson Chi-square values for SEC and Matric grades

computed by gender also confirm this. The only significant difference that emerges is

in 2002 Cohort Assessment 4 of the 2nd^ Year, and this may be attributable to a number

of contingencies such as gender bias. The absence of a trend in gender differences is

also confirmed by the fact that in the various assessments, males and females take

turns at being the better gender. Another thing that is worthy of noticing is that there

are very few differences in the distribution of students’ scores. Generally speaking,

females tend to demonstrate more homogeneous ability than males. However, except

for Assessment 5 of both cohorts, all the values of standard deviation are very similar

to each other. This may be an indication that students are at different stages of

preparation for their ‘real’ assessment, i.e. the Matric.

Table 3. Means, Standard Deviation and t-test values by Gender

2000 Cohort2000 Cohort2000 Cohort2000 Cohort 2002 Cohort2002 Cohort2002 Cohort2002 Cohort

MalesMalesMalesMales FemalesFemalesFemalesFemales MalesMalesMalesMales FemalesFemalesFemalesFemales NNNN NNNN NNNN NNNN Grades andGrades andGrades andGrades and MeanMeanMeanMean MeanMeanMeanMean MeanMeanMeanMean MeanMeanMeanMean SSSScorescorescorescores

S.D.S.D.S.D.S.D. S.D.S.D.S.D.S.D.

Ind.Ind.Ind.Ind.

SamplesSamplesSamplesSamples

tttt----testtesttesttest

NS = Not Significant

S.D.S.D.S.D.S.D. S.D.S.D.S.D.S.D.

Ind.Ind. Ind.Ind.

SamplesSamplesSamplesSamples

tttt---test-testtesttest

NS = Not Significant * = p < 0.05 level 96 63 103 49

SECSECSECSEC^ 3.01^ 3.17^ 3.05^ 3.12 0.96 1.06

0.995NS

1.19 1.07

0.369NS

78 60 120 53

1111 65.06 63.17 64.50 65.28

12.83 12.89

0.305NS

14.10 15.46

-0.322NS

78 60 120 53

2222 60.77 61.50 64.21 63.30

16.79 16.06

0.859NS

16.04 16.20

0.342NS

78 60 120 53

3333 58.21 56.83 66.25 65.19

17.15 18.80

-0.260NS

18.16 17.04

0.361NS

78 60 120 53

42.13 42.50 38.71 37.17

1st Year Assessment1st Year Assessment1st Year Assessment1st Year Assessment

TestTestTestTest 11.21 9.88

0.441NS

7.86 7.53

1.201NS

91 70 120 53

(^4444) 52.64 53.79 51.13 55.47 15.62 14.85

-0.207NS

13.62 12.49

-1.984*

91 70 120 53

45.77 47.57 53.25 57.36

2nd Year2nd Year2nd Year2nd YearAssessmentAssessmentAssessmentAssessment 5555 21.74 20.83

-0.475NS

18.43 12.69

-1.474NS

111 70 118 52

Matric A levelMatric A levelMatric A levelMatric A level 3.85 3.91 3.03 3.35

1.52 1.40

-0.534NS

1.20 1.17

1.617NS

5.3 Paper choice in Physics SEC for the 2002 Cohort

The results here do not stray far from predictions such that students that had chosen

paper A at SEC level outperform those that had chosen Paper B. Naturally the highest

differences are present in the SEC grades as the paper choice is in itself barring

students from obtaining certain grades. However, it is interesting to notice that this

difference is also similarly evident as regards the Matric grades. From Table 4, the

standard deviations respect the frequency of the groups, i.e. they are higher for the

most numerous and vice versa. The last two assessments in the First Year show very

little discrimination between these two groups of students, so much so that in

Assessment 3 of the First Year paper B students obtain a higher mean score than

paper A. However, this may be interpreted in a number of ways. It may mean that

paper B students are better in the topics covered at the end of the First Year, but it is

more likely a distortion introduced by ineffective and incoherent assessment tools.

Table 4. Means, Standard Deviation and t-test values by SEC paper Choice for 2002 Cohort SECSECSECSEC PaperPaperPaperPaper AAAA

SECSECSECSEC

PaperPaperPaperPaper BBBB NNNN NNNN MeanMeanMeanMean MeanMeanMeanMean

2002200220022002

CohortCohortCohortCohort Grades andGrades andGrades andGrades and ScoresScoresScoresScores S.D.S.D.S.D.S.D. S.D.S.D.S.D.S.D.

IndependentIndependent IndependentIndependent

Samples tSamples tSamples tSamples t----testtesttesttest

NS = Not Significant ** = p < 0.01 level *** = p < 0.001 level 125 27 SECSECSECSEC 2.83 4.19 1.12 0.40

10.754***

126 27

1111 67.58 58.52

14.00 12.85

3.272**

126 27

2222 65.56 61.30

16.21 15.85

1.262NS

126 27

3333 67.06 67.22

18.47 17.23

-0.043NS

126 27

39.54 34.67

stststst 1111 Year AssessmentYear AssessmentYear AssessmentYear Assessment

TestTestTestTest 7.72 6.71

3.329**

126 27

4444 53.37 46.11

13.52 9.34

3.357**

126 27

57.06 45.56

ndndndnd 2222

YearYearYearYear AssessmentAssessmentAssessmentAssessment 5555 15.79 17.56

3.144**

124 27

2.90 4.00

Matric AMatric AMatric AMatric A levellevellevellevel 1.09 1.27

4.162***

5.4 SEC Ability groups

Organizing students according to paper choice can provide an idea of whether the

SEC can be an effective predictor of performance in Matric. This possibility becomes

more evident when one divides the students into two groups according to their SEC

grades. The two groups that have been formed are more or less of the same size,

composed of about 75 students each. The first group consists of those students that

obtained grades 1 to 3 in SEC while the second group is made up of grades 4 and 5.

The distribution of grades shows that students that had obtained a grade 1 to 3 in their

2000 Cohort 2002 Cohort

0

5

10

15

20

25

F re q u e n c y

A B C D E F Matric Grade

SEC 1-3; N=90 SEC 4-5; N=65

0

5

10

15

20

25

30

F r e q u e n c y

A B C D E F

Matric Grade

SEC 1-3; N=75

SEC 4-5; N=75

SEC manage to do better than those that have obtained the lower grades. However, it

is also clear from Figure 1 that having obtained a low grade at SEC level is not an

impediment to continuing further studies in Physics. In some cases the low grade in

SEC has been transformed into a grade B at Advanced level.

Figure 1 Distribution of Students’ Matric Grades by SEC Ability group

Table 5 confirms the trends that emerged from Figure 1. The students who had

obtained top grades at SEC level significantly surpass those from the lower grades in

virtually all the assessment exercises carried out in these two years. It is only in

Assessment 3 of the First Year that the difference between the two fades into

insignificance. However, although the difference is conspicuous and consistent it

does not mean that those students with a low SEC grade are failing at Advanced level.

In virtually all assessments they still get a good score and in the Advanced level they

obtain a mean grade of between C and D (3.63) which is still a useful grade to

continue further education and also to seek employment.

Table 5. Means, Standard Deviation and t-test values by SEC Ability group SECSECSECSEC Grades 1Grades 1Grades 1Grades 1 to 3to 3to 3to 3

SEC GradesSEC GradesSEC GradesSEC Grades 4 and 54 and 54 and 54 and 5 NNNN NNNN MeanMeanMeanMean MeanMeanMeanMean

Grades andGrades andGrades andGrades and ScoresScoresScoresScores

S.D.S.D.^ S.D.S.D.^ S.D.S.D.S.D.S.D.

IndependentIndependentIndependentIndependent

Samples tSamples tSamples tSamples t----testtesttesttest

NS = Not Significant ** = p < 0.01 level *** = p < 0.001 level 77 75 SECSECSECSEC 2.10 4.07 0.80 0.25

20.195***

77 75

1111 71.36 60.20

12.61 13.47

5.277***

77 75

(^2222) 68.18 61.13 16.46 15.19

2.741**

77 75

3333 69.48 64.40

18.93 17.18

1.731NS

77 75

40.91 36.20

stststst 1111 YeaYeaYeaYear Assessment

r Assessmentr Assessmentr Assessment

TestTestTestTest 7.79 6.86

3.951***

77 75

4444 56.56 47.47

13.28 11.46

4.513***

77 75

60.13 49.87

ndndndnd 2222 YearYearYearYear AssessmentAssessmentAssessmentAssessment 5555 15.69 16.19

3.969***

75 75

Matric A levelMatric A levelMatric A levelMatric A level 2.57 3.63 1.05 1.11

5.951***

5.5 Matric Ability Groups

The Matric result was used to organize the students into two ability groups, one for

the students who obtained the top grades A, B and C and one for the lower grades D,

E and F.

Figure 2 shows that students who have obtained the higher grades had also obtained

top grades in Physics at SEC level. On the other hand, the vast majority of students,

74% of 2002 cohort, from the lower Matric ability groups had previously obtained a

grade 4. The remaining quarter of the students are more or less equally spread over

the other four SEC grades. A similar distribution occurs for the top ability group but

without there being the distinct peak over a particular grade. The percentage of

students increases on going from grade 1 to grade 4, although this may be attributed to

the lower grades being more numerous in general.

0

10

20

30

40

50

60

70

80

F re q u e n c y %

1 2 3 4 5

SEC Grade

Matric A to C; N=100 Matric D to F; N=50

2000 Cohort 2002 Cohort

0

5

10

15

20

25

30

35

40

45

50

F r e q u e n c y %

1 2 3 4 5 SEC Grade

Matric A to C; N=100 Matric D to F; N=50

Figure 2 Distribution of Students’ SEC Grades by Matric Ability group

Further consolidation of this is shown in Table 6 that highlights the differences

between the Matric ability groups throughout the two years of study. All the

differences are significant, mostly at the 0.001 level. It seems that students that end

up with a low grade in Matric had started off by obtaining a lower grade in SEC and

then continuing to get lower scores throughout their two years of study. However, in

spite of there being this overall trend it does not force the lower ability students

sufficiently down the scale such that they may be considered as having failed their

post secondary education. Learning takes place amongst low achievers too! (sic) The

most consistent predictor of performance seems to be the End of First Year test

especially for the 2000 Cohort, with Assessment 5 coming in as close second. These

general trends are confirmed in both cohorts.

Table 6. Means, Standard Deviation and t-test values by Matric Ability group

2000 Cohort2000 Cohort2000 Cohort2000 Cohort 2002 Cohort2002 Cohort2002 Cohort2002 Cohort

MatricMatricMatricMatric GradesGradesGradesGrades A to CA to CA to CA to C

MatricMatricMatricMatric GradesGradesGradesGrades D and FD and FD and FD and F

MatricMatricMatricMatric GradesGradesGradesGrades A to CA to CA to CA to C

MatricMatricMatricMatric GradesGradesGradesGrades D and FD and FD and FD and F NNNN NNNN NNNN NNNN MeanMeanMeanMean MeanMeanMeanMean MeMeMeMeanananan MeanMeanMeanMean

GradesGradesGradesGrades andandandand ScoresScoresScoresScores

S.D.S.D.S.D.S.D. S.D.S.D.S.D.S.D.

Ind. SamplesInd. SamplesInd. SamplesInd. Samples

tttt----testtesttesttest

** = p < 0.01 *** = p <

0.001 (^) S.D.S.D.S.D.S.D. S.D.S.D.S.D.S.D.

Ind.Ind.Ind.Ind.

Samples tSamples tSamples tSamples t---

testtesttesttest

** = p < 0.01 *** = p < 0.001 58 97 100 50 SECSEC SECSEC 2.62^ 3.34^ 2.77^ 3.74 1.15 0.80

4.188***

1.12 0.90

5.729***

56 75 111 59

1111 69.91 60.47 68.78 57.29

12.77 11.42

4.378***

13.55 13.56

5.262***

56 75 111 59

2222 67.05 57.07 67.34 57.97

15.19 16.01

3.492**

14.30 17.37

3.555**

56 75 111 59

3333 66.34 52.20 69.59 60.34

15.45 16.57

5.024***

16.71 17.76

3.301**

56 75 111 59

49.70 37.13 40.68 33.59

stststst 1111 Year AssessmentYear AssessmentYear AssessmentYear Assessment

TestTestTestTest 9.32 8.20

8.031***

7.47 6.07

6.682***

63 81 111 59

4444 61.43 47.47 55.68 46.78

12.59 12.87

6.537***

13.92 10.33

4.718***

63 81 111 59

59.42 40.00 60.36 43.81

ndndndnd 2222 YeYearYeYe

ararar

AssessmentAssessmentAssessmentAssessment 5555 18.08 15.97

6.898***

15.74 13.59

7.146***

67 110 111 59

2.25 4.81 2.41 4.47

Matric AMatric AMatric AMatric A levellevellevellevel 0.82 0.77

20.508***

0.65 0.73

18.286***

As a summary, one can compare the Matric Ability group with the SEC ability group.

The distribution of values in Table 7 demonstrates how obtaining a good grade in

Physics at SEC level is a good start towards obtaining a good grade at Matric level.

However, getting this good grade is only a rough indicator as there are an appreciable

number of students (50) with low SEC grades (4-5) who manage to get a top grade in

Matric A level. Also, a good number of students (54) with a high SEC grade (1-3)

do not keep up their good performance at Advanced level. Besides these general

trends, there is also an evident difference in the performance between the two cohorts.

Presumably hard work in studying pays more than a good starting grade!

Table 7. Matric Ability group compared to SEC Ability group

Matric Ability GroupMatric Ability GroupMatric Ability GroupMatric Ability Group

SEC Ability GroupSEC Ability GroupSEC Ability GroupSEC Ability Group

HighHighHighHigh LowLowLowLow TotalTotalTotalTotal

HighHigh HighHigh 42 48 90

LowLow LowLow 16 49 65

2000200020002000

CohortCohortCohortCohort

TotalTotalTotalTotal 58 97 155

HighHigh^ HighHigh^66 6 72

LowLow^ LowLow^34 41 75

2002200220022002

CohortCohortCohortCohort

TotalTotalTotalTotal 100 50 147

HighHigh HighHigh 108 54 162

CombinedCombinedCombinedCombined LowLow LowLow 50 90 140

TotalTotalTotalTotal 158 147 355

5. Conclusions and Recommendations

There are fewer females than males pursuing the Physics Advanced course at the

Junior College. From 2000 to the 2002 there is a pejorative progress; the male to

female ratio falls from 1.5 in 2000 to 2 in 2002. Furthermore it appears that an even

less proportion attempt the Matric Physics Advanced nationally. In 2002 the ratio of

2.5:1 was calculated using the relative frequencies for the entire Maltese Physics

Advanced cohort (from frequency data MATSEC 2002 Matriculation Statistical

Report). This may indicate that the Physics Advanced College course attracts more

females to pursue studies in the physical sciences than the other post-secondary

colleges although still more needs to be done as regards the gender gap. This is in

sharp contrast with the fact that more females than males sat for the SEC 2000.

Furthermore, in a previous study, it had been found that there are no significant

gender differences in performance in the Physics SEC 2000 and only slightly better

grades obtained by females (Pace, 2002b). This study also shows that that there are

more females than males opting for the SEC Paper B (ibid.) In the short space of two

years, the similar ability demonstrated at SEC level between the two genders is

replaced by a male superiority in numbers and in results.

This evidence derived from the two Physics examinations indicates the loss of a good

proportion of females continuing further studies and subsequent professional careers

in the physical sciences or related. This could also mean that the Physics Advanced

course does not have a sufficient intake of the Physics SEC high ability students.

Further indication of the lack of females pursuing higher education in Physics is the

low standard deviations both in the SEC and Matric examinations shown in this study.

These indicate that these females are a narrow range of high achieving students and

that the average ability female students for some reason or other are opting out of

continuing Physics at Advanced level.

Identifying valid predictors for Matric Advanced level grades is never straight

forward. For one thing, not one of the summative or formative assessments employed

from SEC to the Junior College internal tests and scores correlates that strongly with

the Matric Advanced level results. Therefore, obtaining a top SEC grade or doing

well in the Junior College Assessments is not a sure recipe for success at Matric level.

However, the correlation values (Table 2) between SEC and Matric are still indicative

that these students may be already on the right track from the start. This correlation

coefficient, although rather low, is not unimportant as it is a measure between two

examinations that are essentially different in format and skills tested, concepts learned

and medium used, not to mention difficulty level. One important difference between

the two examinations is the Practical component which is formally examined at

Matric Advanced level while at SEC level it is measured using school based

continuous assessment. Furthermore, the mathematical/analytical content is

drastically increased at Matric level as is the level of English employed. The format

of the paper also changes and the students are given a lot more choice at Advanced

level when one compares it to SEC which offers no choice of questions at all. In a

similar study to this one, Ventura (2001) calculated an r-value of 0.572 for SEC 1998

and Matric 2000 cohort that is higher than the ones obtained here but still indicative

that the two Physics external examinations are essentially different.

This low predictive validity of the SEC grades, or any other assessment tool

considered for that matter, also suggests that with hard work and commitment at the

Junior College, a student that starts off with a low SEC grade stands a 50% chance of

obtaining a good grade in Matric Advanced level. If not a good grade, but a useful

grade to continue further education is certainly within their reach and so even a low

paper-B SEC grade 5, is still within the margins of success for a useful Matric result.

This should be of credit to the Physics tuition at the College apart from the efforts

done by the student. In the light of this study it would definitely not be advisable if

the Physics course entry requirements were made more stringent. As argued in

Section 5.5 above, (Table 7) restricting entrance to Physics Advanced to those who

obtain grades 1-3 in Physics SEC, implies rejecting about a third of the cohort who

attain low SEC grades (4-5) but succeed in getting the higher grades A-C in the

Advanced while at the same time keeping another third of those that achieve 1-3 in

SEC but who only manage to obtain grades D and E in Advanced level. If assessment

is more attuned to diagnose student difficulties and assess their potential, then a

programme can be implemented whereby students may be guided better to achieve

optimum results.

This area certainly requires further study. A cursory analysis of drop outs from first

year to second year in 2002 reveals that about 90% of them had indeed chosen Paper

B for their SEC examination. Therefore, this may mean that only paper-B SEC

students with a characteristic resilience manage to make it through the two years of

study, to eventually have a chance at the Advanced level. The success of the paper-B

students may thus be attributed to the weeding out of the really weak ones during the

first year and the consolidation of the resilient types. However, it may also mean that

students who are dropping out of Physics Advanced level course are not receiving the

proper support. This fact calls for a lower student-lecturer ratio during lectures that

presently stands at around 50:1 and certainly, and more importantly, a lower ratio

during tutorials presently at about 25:1. The numbers are far too big to successfully

identify and direct students as is merited. The situation does indeed improve during

second year when the effect of the drop outs, harmful as it may be to them, is

beneficial to the ones that remain. Student-to-lecturer ratio falls slightly, the

assessment tools become much more effective and the quality of the learning

increases.

A very important indicator of performance was the end-of-first-year test for the 2000

cohort which contrasts highly with the one for 2002. This difference seems rather

peculiar when one considers that the results of this study show such comparability

between the two cohorts. However, there are significant differences between the two

end-of-first-year tests. The method of construction of the test has changed from the

first to the second instance. For the 2000 cohort, the test was similar in format to the

Matric in that it consisted of two written papers and a practical component. This was

constructed by means of a group effort of all the lecturers at the Physics Department.

On the other hand, the 2002 cohort were tested by a single written paper and practical

constructed solely by the Physics Subject Coordinator and reviewed by two lecturers.

These two differences alone may account for the discrepancy in correlation between

the End of First Year test and the Matric for the respective cohorts.

This study puts into emphasis that the informal assessments throughout the Junior

College Physics Advanced course have a more formative function and so they may be

used to diagnose those students that are falling behind. As students progress in their

two year course, so does the efficiency of these assessments in predicting success or

failure in Physics Matric A level with the notable exception of the End of First Year

test in the 2000 cohort. This culminates in Assessment 5, just before students sit for

their Matric that shows the greatest degree of commonality out of all the assessment

tools analysed with the Matric Advanced level result. This suggests that by this time

students have gathered a certain momentum in their studies, having mastered

concepts, practised skills and gained insights. Although the rather high standard

deviations indicate that some students are still not at the top of their preparation.

Assessment 5 should be considered by students as an indication of whether they are

on the right track or whether they need to work harder still. However, what the

argument outlined in the previous paragraph shows is that perhaps a mock test would

give more valid feedback to the students. Thus, they would be able to bridge the gap

between the fuzzy knowledge that they obtain with the present system to a coherent

account of strengths and weaknesses ensuing from an examination.

References:

Bakker, S & Wolf, A. (2003): Upper Secondary Examinations and Entry to University: The School: University Transition in an Age of Mass Education , in Assessment in Education: principles, policy and practice, Vol. 8. Number 3, Taylor and Francis, London.

Cam, P. (2003): The French Baccalauréat Since 1985: Level of Qualification or Type of Diploma? , in Assessment in Education: principles, policy and practice, Vol. 8. Number 3, Taylor and Francis, London.

Camilleri, R.,[ed] (2005) The Lisbon Objectives and Maltese Education Provision. Conference Proceedings: Rising to the Challenge Malta 2004, Education Division – Ministry of Education, Youth and Employment, Malta.

Douglas Willms, J. (2003), Student Engagement at School: A sense of Belonging and Participation – Results from PISA 2000 , OECD.

Education Division Malta (2000): Guidance and Counselling Services: Description of Services Manual, Education Division, Malta.

Gipps, C. (1990): Assessment: A Teachers’ Guide to the Issues. Hodder & Stoughton, London.

Grima, G., Camilleri, R., Chircop, S & Ventura, F. (2005), MATSEC Strengthening a National Examination System , Ministry of Education, Youth and Employment, Malta.

Gronlund, N.E. (1987): Measurement and Evaluation in Teaching. Macmillan Publication, New York.

Junior College (2000a): Junior College Strategic Plan 2000-2005. University of Malta, Malta.

Junior College (2000b): Look before you leap the Junior College Prospectus. University of Malta, Malta.

MATSEC (2002): Matriculation Certificate Examination 2002 – Statistical Report. MATSEC Board, University of Malta, Malta.

Nardi, E. (2003): The transition from school to University in Italy: Examination reform and outstanding issues, in Assessment in Education: principles, policy and practice, Vol. 8. Number 3, Taylor and Francis, London.

Pace, E. & Pace, J. (2002): School-Leaving Population Sitting for Physics SEC 2000 examination and subsequent Science Related Career Choices. A paper presented at the CASTME S & T Education Conference, Malta -10th-13th^ April 2002.

Pace, J. (2000): The Differentiated Paper System in the Physics SEC Examination. Unpublished M.Ed research paper, Faculty of Education, University of Malta.

Pace, J. (2002a): Fairness and Gender Equity within the Differentiated Paper System in Physics SEC Examination in Malta. A paper presented at the 2nd ACEAB Conference, 18-22nd March 2002, Malta.

Pace, J. (2002b): The Physics SEC 2000 Examination: a focus on the Differentiated Paper System. Unpublished M.Ed dissertation, Faculty of Education, University of Malta.

Pace, J. (2003): The Differentiated Paper System in Maltese SEC Examination: Is it promoting Quality, Equity and Fairness? A paper presented at the BERA 2003 Conference, 10-13th^ September 2003, Edinburgh, Scotland.

Ventura, F. (2001): The Predictive Validity of the SEC Examination for A level Sciences: Some preliminary results. Paper presented at Science Education Seminar, Malta 23rd^ May 2001.

Zarb Adami, M., Debono, S. & Sammut, F. (1999): The MATSEC Examinations: A Report by the MATSEC Analysis Facilitating Board. Unpublished Report, Ministry of Education, Malta.