We need real Subject Matter Expert and In-House External Examiner
All educators and academicians in any educational institutions will face the subject matter expert, in-house external examiner, ‘academic secretary’ and not forgetting the head of department to ensure that the course work and also the examinations paper will be endorsed for use.
In normal practice, the teaching team consisting of ‘educators’ of the taught subjects will be assigned questions or items to build based on the taxonomy of learning and the ‘table of specification’ provided by the curriculum and also based on the ‘course academic program’.
Right after the items or questions and the answer schemes are constructed and built by the ‘academics’ they will be handed to the respective ‘SME’s to the ‘head of department’ and to IEE’s for endorsement plus the ‘academic secretary’ before being handed to the Examinations Unit for printing.
There is always tension present during the ‘building’ stages especially in the selection of standard-setting participants. The ‘educators’ are the one who teaches the subjects, the ones who knows what’s going on in the classroom, who knows what the students had learned so far and ‘supposedly’ know what to expect from the papers ‘built and constructed’ by.
But when the ‘proposed papers’ were rejected till 6-7 times by the higher authorities consisting of the ‘SME’s’, the IEE’s and the ‘head of department’ before being accepted there must be something awkward happening along the process. What’s actually wrong and disturbing about the papers build by the educators?
A question asked, do we need SME/IEE at undergraduate level, whereas, elsewhere SME/IEE are at postgraduate level? Okay, quality comes first, some people said.
A subject matter expert (or SME) is someone who knows their stuff inside out, with bona fide expert knowledge about what it takes to do a particular job.
The in-house external examiners should be a subject specialist who acts as a defender of the standard, principally establishing whether or not a particular course is delivering academic standards broadly in line with those of other higher education institutions.
While the ‘head of department’ should be supporting his/her educators to ease the process of item building and construction.
That’s exactly how a content knowledge people should be able to sound: like someone who knows how to help your people get stuff done.
When too much repeated and re-doing of work was given. It is most likely that the ‘SME’s’ and the IEE’s should be doing the ‘course work and the exam papers’, after all they are the ‘experts’ based on the given title to them. If our doings are ‘wrong and not correct’, why not let the ‘experts’ finished up the work.
How are the SME’s, the IEE’s were selected or elected to the post? SME’s and IEE’s will most likely already exist within a practice but may need to be identified and nurtured by the practice leaders.
They should likely have proven work of writings and researches, authors of academic books and presenters of locally and international papers but most are ‘picked’ from the pools of people.
Likely questions came up that they do need to have PhD but good enough if they are ‘experts in the field of studies’. But can an English trained SME/IEE be picking on ‘Educational Studies’ or ‘Visual Arts’ subjects?
Though effective item writers are trained, not born and the ‘teaching team who is also the item writers’ are the people in their own disciplines, there is no reason to believe that their subject matter expertise generalizes to effective item writing expertise.
No doubt and often there will be errors or inconsistencies in the content, which can be clarified by item authors or item content reviewers. Thus SME/IEE can assist in content reviews, and editorial review, which will strengthen the content-related validity evidence for the test.
Diverse Input Involve people with diverse points of view in the test development process to help ensure that the resulting test is appropriate for all of the intended test takers. It has become common practice to use committees of external subject matter experts to help with test design, item writing, and item review.
The SME/IEE’s should know what they are talking about as they are required to have a high level of domain knowledge. They should also know who they are talking to as they cannot operate on assumptions. They must know how to talk to the original ‘item builder’ so as to ensure that the conversion can reach them.
Many educators or the ‘item writers’ want SME/IEE who are supportive and critical friend. Many would expect friendly SME/IEE and not the cozy type.
Importantly it demonstrates external examining as a two-way street and not just one-way transfer of ideas from a visiting expert to the host institution.
The SME/IEE should provide a balanced view and draw attention to good practice as well as any problems.
A well rounded feedback plus commentary should be most welcome.
Documentation that describes the process and results of these activities should be available and should reference published materials that potential examinees can access.
As part of this documentation of the content specification processes, the test developer should describe the qualifications of the subject matter experts (SME) and also describe the extent to which they participated in the process.
To evaluate whether the SMEs were appropriate, an evaluator needs to know who these individuals were and what they did during the development processes.
Azizi Ahmad is an educator Wed, Mar 27, 8:34 AM