Design of assessment, grading, and delivery

Return to index

The procedures to develop assessment methods for individual units and groups of units

• Chief Assessor sets out the assessment design parameters for units as follows:
  • Assessment unit title that
    • clearly and concisely reflects content of the unit
    • Is self-standing and does no make reference to external information
    • indicates the level of attainment expected
  • Assessment criteria that
    • are presented in language that is accessible to assessors and learners
    • are directly related to the unit learning outcomes and link to them on a one to one basis
    • can be directly matched to a learner's knowledge, understanding, and practical capability
    • are at the approriate level matching the RQF/CQFW level in which the unit is situated
    • are independent of assessment in any other subjects or units
    • reduce the likelihood for reasonable adjustments to be applied at a later date
  • Guidance on interpreting the criteria that
    • form a clear basis as to how assessors should interpret the criteria and the degree of detail and depth required
    • links to illustrative examples that will help establish consistency in the approach to learning
  • Verification and moderation methods that provide a valid and reliable means of judging capabiliy against the criteria
    • with minimum bureaucratic burden on assessors and learners
    • that encourages learning and involvement of the learner in the assessment process
    • that feeds back positively to the continuing professional development of assessors
    • that is matched to the level of attainment in terms of complexity and demand.
  • Design is fully compatible with the specification defined by Section 1 of the Regulatory Arrangements

• Chief Assessor assisted by account managers define specific assessment criteria that conform to the design parameters.
• Assessment criteria checked by independent person with appropriate expertise and then signed off by The Chief Assessor
• Assessor guidance produced to ensure that sufficient evidence is gathered by assessors to make a valid judgement as to when the criteria have been matched
• Assessor guidance informed by expected levels in similar qualifications at similar levels in the RQF/CQFW, NDAQ and National Curriculum
• Assessor guidance includes areas where reasonable adjustments might occur and ensures  these are eliminated by making the assessment solely dependent on specific learning outcomes associated with the subject matter.
• Assessor guidance checked by independent person with appropriate expertise and then signed off by Chief Assessor
• Draft moderation and verification methods devised to match the type of learning eg Multiple choice tests for knowledge, e-portfolio for practical production of digital resources, focused practical tasks for operating software.
• Draft methods checked by independent person with appropriate expertise and then signed off by Chief Assessor
• Assessment materials/instruments made available from the community site in finalised form with links to guidance, exemplars and supporting materials
• Handbook revised to include new units and assessment methods
• Check that units and assessment methods conform to the format and design requirements of the qualifications regulators given in the RQF/CQFW Regulatory arrangements publication including diversity and equality of opportunity and that any units can be assessed individually.

Centre based assessment requires assessors to directly match assessment with the objective criteria using a range of valid and appropriate methods that

• Ensure coverage of the criteria by requiring all criteria to be matched at least to a specified minumum level.
• Reflect the levels of the RQF/CQFW by relating the criteria directly to the unit learning objectives.
• Encourage and motivate learning by supporting individual target setting self and peer assessment prior to assessor final judgements.
• Provide sufficient evidence to enable reliable and consistent judgements to be made about attainment in relation to the assessment criteria by providing guidance on assessment details and expected levels for all criteria and training assessors accordingly.
• Minimise costs and administrative overheads for centres and their learners by providing free and open learning resources and on-line learning platforms in support of the qualifications.
• Check that there are no steps that can be taken to minimise the subsequent need for reasonable adjustments or similar access arrangements.
• Feedback from Centres is acted upon within the constraints of the regulatory framework to maximise the benefit to learners and minimise costs to centres.

Grading

As part of the process for the development of assessment given in 5.3 above, where appropriate, grading will be assigned with checking against the design features laid out in the RQF/CQFW Regulatory Arrangements (Section 1). The judgements about the grading of qualifications and their components will be made using the procedures outlined below.

• For Entry level units there will be no grading, simply a pass level when the assessment criteria are matched by the learner performance.
• Pass grades will be benchmarked against Level 3 in the national curriculum.
• For level 1 and level 2 units grading will be in keeping with the sector qualifications strategy and agreed with e-skills. 
• Qualifications will be graded on the following principles. To pass a unit all criteria must be assessed as secure or higher. Any supporting evidence or verification tasks must be completed to the satisfaction of the Account Manager at TLM. For a pass in the qualification sufficient credit must be gained from the required units. Grades above a pass will be determined by additional criteria, and/or achieving a minority of units from a higher level.

Delivery of assessment

Assessment is delivered using a combination of assessors and independent verifiers (The Senior Account Manager and Account Managers are full-time verifiers employed by TLM. Their key role is to provide external independent checks on the work of assessors using the internet and visits to the centres.) Each Centre must have a trained Principal Assessor who takes responsibility for assessment standards within the Centre. The Principal Assessor must sign a contract with the Awarding Organisation agreeing to uphold standards and implement quality assurance procedures in keeping with the RQF/CQFW Regulatory Arrangements. The Principal Assessor can delegate assessment to authorised assessors within their Centre who make assessment judgements against the criteria using a range of assessment instruments provided by the Awarding Organisation (TLM). Each assessor has a named Account Manager at TLM with whom they can build a working relationship. The Account Managers at TLM sample evidence related to the judgements of the assessors to provide external and independent quality assurance. The Account managers are overseen by The Chief Assessor with input from the Principal Moderator who represent the Governing Body. The outcomes of sampling are fed back to the assessors to inform their own professional development and to enhance the achievement of consistency.

1. The procedures for developing assessment instruments are provided in 5.3 above. These procedures are designed to ensure that assessment methods and the supporting instruments of assessment are produced to the required quality. The Principal Assessor signs an agreement that includes the requirement that all their assessors ensure that work is authentic. Candidates are involved with this process through peer evaluation and in some coursework elements making a declaration that their work is genuinely their own. In formal testing candidates are required to perform the tests and tasks in controlled conditions with no communications between themselves and their peers. External verification checks are used with random sampling to further reduce the chances of plagiarism

2. Standards over time are monitored through the moderation procedures. In cases of assessment through controlled tests and tasks samples taken for moderation and verification are stored. These can be used to monitor consistency of standards across units and over time. We review these annually and feedback relevant information to inform assessment design. Assessors receive regular feedback from Account Managers as the moderation and verification process is designed to promote consistency through regular communication between Account Managers and Assessors. E-portfolio work is continually available as it is built up and is stored on line. These portfolios provide exemplars of good practice for future learners to follow and enable monitoring of standards over time and across relevant units. One item of note is that The Learning Machine certificates are base lines of competence and many students are expected to operate at the baseline +. This does not necessarily mean that they have reached the next certificate level as this will require breadth as well as increased capability in particular aspects of IT. What matters over time is that the baseline where a pass is allowed is consistent and therefore the main area of interest will be on evidence at the borderline of achieving the certificate.

3. The people involved in assessment are the assessors in the Centres and the Account Managers, Chief Assessor and Principal Moderator at the Awarding Organisation. The assessors are all trained and are usually but not exclusively qualified teachers. The details of training for account Managers and Expertise of Awarding Organisation personnel can be found in the document Training Programme for Account Managers. The company appraisal system, the process of checking the development of assessment instruments and training of assessors ensures that those involved in the assessment process have appropriate expertise and are adequately informed to fulfill their responsibilities.

4. Training is provided for Awarding Organisaion personal as detailed in the Training Programme for Account Managers. All assessors must be trained by an accredited assessor trainer and all assessor trainers are trained by the Chief Assessor. The policy and procedures for assessor training are provided in Training of assessors and assessor trainers.

Performance management

There is an established appraisal system for TLM staff. Account managers monitor the performance of the assessors using the moderation and verification procedures to provide feedback. The overall performance of the Centre is discussed at the Annual meeting.

Retention of evidence of learners' work

Learners' work for all qualifications is retained in total for e-portfooio work and samples of a minimum of 10% for Awarding Organisation set tests and tasks. This evidence can be used to monitor standards over time.

RPL

As long as there is clear evidence that the learner can match the assessment criteria an Award can be made based on prior learning. The Awarding Body is only making Awards based on evidence of learning outcomes and so learners can be accredited at any time when ready in keeping with RPL and the personalised learning agenda.

Exemption

Any claim for exemption that can be supported by convincing evidence will be upheld.

Reasonable adjustments

Reasonable adjustments, special considerations and Aegrotat Awards. The policy and procedures for reasonable adjustments, special considerations and Aegrotat Awards is given in the document Reasonable Adjustments Policy.

Conflicts of interest

The TLM quality assurance model requires assessors in Centres to be under the scrutiny of Account Managers at the Awarding Organisation. This ensures that there is always an independent view of the standards being certificated. There is more detailed information about quality assurance in the document Procedures for Quality Assurance

Standardisation across centres and awards

5. The administration of quality assurance is identical for each Centre. Principal Assessors ensure consistent standards across their Centre under the scrutiny of TLM Account Managers. TLM Account Managers deal with up to 100 Centres each and they are in turn under the scrutiny of a Senior Account Manager and the Chief Assessor with input from the Principal Moderator.

6. The procedure for checking criteria matching is for the Principal Assessor in the Centre to implement a quality assurance policy as provided in the contract with the TLM on the certification site. This will be verified at the annual visit. The criteria matching process is verified externally using Awarding Body set tests and tasks and sampling of on-line e-portfolios and Blogs.

7. Where relevant, the sampling of Awarding Organisation tests, tasks and on-line work will involve identifying candidates that have recently had results from a test or task entered into the on-line mark book on the certification site. The Account Manager will ask for unpredictable samples representing a spread of attainment and they will check the marking against the marking schemes for each task.  A sample size of at least 10% will be used but in any case the size will be statistically meaningful, for example if the entry size is 5 or less all the work will be checked. Where there is a difference of opinion between the Account Manager and the Assessor, the Account Manager will contact the Assessor and explain the reason why marks need adjustment. They will then request that the Assessor adjust any other marks similarly affected and the mark book entries will be refreshed. The Account Manager will keep a record of all marks that have been changed as a result of the moderation/verification process. In the case of an appeal against the Account Manager's decision, the appeals procedure will apply.

8. The Account Manager's work will be monitored by the Chief Assessor who will meet with the Account Manager monthly and scrutinise records sampling at least 10% of any amendments made to marks.

9. No Account manager will deal with any organisation in which they have a personal interest. All Account Managers are required to declare formally to the Chief Assessor if they are aware of any reason why a conflict of interest might arise from them checking the work of any Centre or Academy assigned to them.

Practical Assessment

In the case of practical assessment to judge physical competence of users against a set criteria, reading and general language capability should not be the limiting factor. To this end for Entry level and Level 1 Awarding Organisation set tests and tasks that require reading questions and information, Centres can read questions to candidates. Every attempt should be made to support simple plain language.

The context of questions and tasks is considered to ensure they are generally applicable and known to the full range of users, for example age, male/female and culturally neutral.

The conditions for delivery of AO set tests and tasks and for coursework are provided on the community site and are part of the training provision. The Assessor requests a test/task from the certification site mark book when the criteria matching has been completed to the appropriate level. Account manager assures that the test/task is sent out and that it is not a test/task that could have been seen before by the candidates. Assessor administers the test and task transferring marks to the mark book. Account Manager samples work providing feedback as appropriate Account Manager together with Chief Assessor makes awards.

Return to index