Procedures for Quality Assurance

Return to index

1. Introduction

These procedures should be read in conjunction with the Handbook and the Academy Centre Contract on the certification site. The key procedure is to establish good communications between the Principal Assessor in the Centre and the Account Manager at The Learning Machine. E-mail should be the chief means of communication but other means are acceptable depending on particular circumstances. Account Manager E-mails are of the form firstname.secondname@TLM.org.UK   or  firstname.secondname@theingots.org (used to be firstname.secondname@thelearningmachine.co.uk which will still work)

2. Fundamental Principles

All awards will be made by Account Managers or other authorised personnel acting on behalf of the Chief Assessor at The Learning Machine Ltd. The personnel at The Learning Machine Ltd will not deal with any Centre or Academy where they have a particular personal interest and will declare such interests as soon as they are known. No member of The Learning Machine Staff will take part in any direct assessment of learners, they will confine their activities to quality assurance of assessors work. No assessor will take part in any moderation or verification activity.

3. Quality assurance is dependent on 2 key factors:

1. The assessors' understanding of the assessment criteria and how they relate to the minimum competent performance for an award.
2. The assessors' ability to make consistent judgements against the criteria interpreting the appropriate minimum level of competence by matching performance to the criteria. Assessors must guarantee that evidence produced by candidates is their own work. This quality control measure ensures that all centres, irrespective of the qualification offered, have continuous and ongoing moderation via external sampling, with certification for learners only available once this quality assurance process has been completed.

4. Important factors that can hinder quality assurance include:

• Assessors who do not have the appropriate technical knowledge and skills about which they are making judgements
• Assessors that interpret the criteria too harshly or too leniently
• Over-complex administrative systems that distract from the main focus and lead to errors, for example, in transfer of data
• Willful malpractice
• Ineffective security

5. Procedures directly supporting quality assurance:

The following procedures are designed to support consistent high quality.

1. All assessors must be trained by an assessor trainer that has in turn been trained by the Chief Assessor
2. No person will be granted assessor status without clear evidence that they understand the assessment criteria and the guidance for assessors in the handbook
3. No person will be granted assessor trainer status without satisfying the Chief Assessor that they understand the assessment criteria and the guidance for assessors in the handbook and that they have the capability of making sound judgements about others in relation to meeting the assessor criteria
4. All active assessors will provide evidence of their assessments in accordance with the requirements specified for each award in the handbook and guaranteeing that evidence presented is the candidates own work and that it has been judged in the context of general level statements as well as the assessment criteria for a specific award. 
5. Where candidates provide evidence from a group activity the assessors judgement is based on that individual's contribution. 
6. Where conditions or controls are required for an assessment these are properly and appropriately implemented.
7. The Principal Assessor will establish quality assurance within the Centre and will provide details of the Centre's procedures eg through an annual standardisation meeting on the Certification Web site as part of the Centre contract with The Learning Machine.
8. There will be an annual audit agreed by the Principal Assessor and the Centre Account Manager at The Learning Machine Ltd for the purpose of moderation and quality assurance.
9. The Centre will provide evidence to the visiting Account Manager in accordance with the requirements detailed in the handbook.
10. Dialogue between account Managers and assesors with regard to verification of work samples is used as a tool for moderation to ensure consistent interpretation of the criteria across Centres.

6. Procedures for monitoring and increased monitoring

The normal level of monitoring will be the annual audit by the Account Manager. In instances where an assessor's judgement is in doubt based on the evidence provided from moderation or a complaint, either to the Principal Assessor or directly to the Account Manager, the Account Manager will inform the assessor and the Principal Assessor that increased monitoring will be required. The Account Manager will arrange a time to visit and observe the assessor at work (The Principal Assessor can also attend if the Centre wishes). The first such additional visit to a Centre will be free of charge in any Academic Year. Subsequent visits will be charged at £350 per visit to cover costs. If after the first visit, the assessor fails to meet the required standard, they will be suspended as an assessor until such time as they can meet the standard. In the case where the assessor is the Principal Assessor, the Centre must provide an alternative suitably trained person to take on the role of Principal Assessor in order to carry on in the programme. No refunds will be given of Centre or Academy fees if the Centre chooses to drop out of the programme.

7. Removal of Assessor or Centre Status

• The following grounds will be reasons for revoking Assessor and/or Centre status:
• Malpractice likely to bring the programme into disrepute
• Negligence in maintaining password security
• Failure to pay fees to the agreed deadlines
• Persistent failure in making judgements that meet the required standards and lack of evidence that learning has taken place

In all cases there will be a period of suspension of 3 months to allow time for an investigation to take place and appeals will be allowed. (See the Appeals Procedures in "Procedures for Making Enquiries and Appeals") In all other cases additional monitoring will be applied until such time that there is full confidence in the consistency and reliability of the assessor's and Centre's work.

8. Centre quality assurance statement

There follows a specimen quality assurance statement.

Centres can adopt this as it is and the Principal Assessor can post it in the relevant place on the web site with the agreement to be responsible for standards when they log in. The statement can be edited to suit particular circumstances as long as in essence the facility to support maintenance of consistent standards is secure. The internal quality assurance procedures of the Ingot Centre led by <name> are as follows.

The Principal Assessor will formally observe the work of each assessor trainer in the Centre on a minimum of one occasion each year. Each assessor will have their work formally observed by an assessor trainer on a minimum of one occasion per year in keeping with the Centre's existing internal quality assurance procedures. The outcomes of the observation will be recorded in summary together with any specific points of action. Where points of action are related directly to maintaining assessment standards, the action points will be evaluated by a specific date and the outcomes summarised in writing. In the first instance, the Principal Assessor will arrange further training for any assessor that provides any cause for concern in meeting the requirements for consistency. The Principal Assessor will review any action points and outcomes and will inform the Account Manager at The Learning Machine Ltd in any cases where there is a risk that standards could be inconsistent or uncertain. In such cases the Principal Assessor will keep samples of work or written commentary on observation as evidence of the issues involved. The Principal Assessor will convene an annual meeting of assessors within the centre to share ideas related to assessment and to ensure consistency in making judgements about levels. A summary of the findings and any action points will be shared with the Account Manger from the Learning Machine at the annual moderation meeting and recorded on the web site. The Centre will keep evidence from its quality assurance to monitor its own standards over time and will provide evidence on request to The Learning Machine to contribute to the monitoring of standards over time for the whole programme. The Principal Assessor will take responsibility for this work. The Centre will keep basic statistical evidence such as the number of candidates each year achieving each of the levels and the number of candidates each year progressing between levels. This will inform judgements about institutional performance and improvement in line with existing policies across subjects.

9. Monitoring company performance 

The products of The Learning Machine Ltd are services to its Centres. The best indicator of company performance is the satisfaction of its Centres with the service provided. To this end data is collected by 3 key methods

1. Enquiries and other communications between the Centres and The Learning Machine Ltd
2. Posts to the forums
3. An on-going survey of customer satisfaction

All enquiries and communications are logged and summarised in the Centre's files in the database. Posts to the forums and e-mails are monitored and suggestions are used for some immediate changes to procedures where these improve efficiency without affecting quality assurance. The survey of customer satisfaction is available at all times on the login at the certification site. The evidence from this information is used by the Chief executive to compile a summary report to the Certification Board with recommendations for improvement. The Certification Board approves or declines any recommendations for change based on the capacity of the proposed changes to improve efficiency and effectiveness and strengthen quality assurance.

Please refer to customer care policy, policies and procedures for making enquiries and the Handbook for further relevant details.

10. Centre quality assurance statement

The quality assurance statement is as stated above.

Terms of reference of committees or groups responsible for market research, identification of demands, stakeholder liaison.

The Certification Board is responsible for the overall quality of the qualifications. Quality starts with fitness for purpose and that means matching the design to the needs of the market. To this end the Certification Board constitution mandates that the membership of the board is representative of stakeholders including industry, community, assessors and teachers. The Certification Board delegates detailed information gathering to the Chief Assessor and Principal Moderator. Their remit includes participation in networks where market information is currently formulated from the most up to date data and from direct evidence from stakeholders. The Certification Board gives a high priority to ensuring that its officers establish appropriate relationships with sector skills councils, learning and skills councils, professional associations and user groups. At the time of writing, the Chief Assessor is a member of the Awarding Body Forum of e-skills, the sector skills council for IT and telecommunications, a member of the National Association for Advisers in Computer Education, and a member of the Open Source Consortium of companies, Schoolforge UK and BECTA funded Open Source Schools. The Principal Moderator is a Fellow of the Chartered Institute of Educational Assessors. Market research for currently accredited qualifications includes work supported by The British Chambers of Commerce and the EU Lifelong Learning programme.

The Executive Board is responsible for ensuring that the commercial model for new qualifications is viable and that there will be sufficient income to offset development costs. The Chief Executive Officer and Finance Director report monthly to this board in order to monitor financial progress. The Chief Executive works with other members of the board on market research and expansion of the business for which there is a comprehensive though commercially sensitive plan. A summary presentation is at http://docs.google.com/present/edit?hl=en_GB&id=afpcfxgvwcdh_15d5fxqhcb

11. Organisation chart detailing the posts with responsibility for quality assurance.

The company organisation chart is provided in the document Management chart, administration and governance. All members of the organisation and associates have a responsibility for quality assurance.

The Chief Assessor is responsible for the overall quality of qualifications from unit design through rules of combination to assessment instruments.

The Principal Moderator is responsible for independently checking the work of the Chief Assessor and Accounts Director and reporting back to the Certification Board with reference to the overall quality of procedures and any associated risk in assessment methods.

The Senior Account Manager is responsible for ensuring centres comply with TLM requirements including sampling evidence, making visits, devising assessment instruments and providing training. The Accounts Director is accountable to the Chief Assessor and meets regularly with him to review standards and procedures. The Accounts Director oversees the work of Account Managers.

Account managers assist the Accounts Director in their duties and are responsible for work of high quality but all their outputs are checked by at least the Accounts Director before it goes live.

The Technology Director is responsible for high quality support systems to enable low cost and efficient qualifications that are secure and reliable.

The Director of Administration is responsible for the quality of financial monitoring reporting directly to the Executive Board, making returns to the regulators and other stakeholder agencies, gathering relevant information to inform quality assurance such as enquiries and appeals. 

Principal Assessors are responsible for quality assurance across their Centre/Academy, they are accountable to their Account Manager at The Learning Machine.

Assessors trainers are responsible for training assessors in their Centre/Academy. They are accountable to the Accounts Director at The Learning Machine.

Assessors are responsible for the quality of the assessments they make and they are accountable for this to their Account Manager at The Learning Machine.

12. Procedures for ensuring on-going compliance with Regulatory Arrangements

Sources of support

The Awarding Organisation's policies and procedures, as laid out on the community web site, have been designed to support the Regulatory Arrangements and are derived directly from them. This provides the resource base to support on-going compliance with the Regulatory Arrangements. Policies and procedures are referenced to the Regulatory Arrangements from the index at http://www.theingots.org/community/node/5485.

Procedure to ensure on-going compliance

• Biannual review of systems and procedures to ensure they are compliant with current regulations. This will involve
  • Director of admin. reviewing managment systems and procedures including equality and diversity against the most up to date version of the Regulatory Arrangements.
  • Senior Account Manager reviewing systems and procedures for awarding qualifications against the most up to date versions of the Regulatory Arrangements.
  • Chief Assessor assessor reviewing systems and procedures for design of units and rules of combination against the most up to date versions of the Regulatory Arrangements.

Each of these will report to the Chief Assessor who will identify actions based on the evidence provided. Actions and recommendations will be presented to the Certification Board. The Certification Board will accept or reject the recommendations. If the recommendations are rejected, the Chief Assessor will make such revisions as are necessary to satisfy the Board.

These procedures will contrbute directly to the data required for Ofqual's self-assessment requirements.

These policies and procedures will be reviewed biannually against the most up to date version of the Ofqual publication "Regulatory Arrangements for the Qualifications and Credit Framework" and amendments, improvements and updates implemented as required. The web pages holding TLM policies and procedures are cross-referenced to the regulatory arrangements and the outcomes of review used to inform self-assessment with the outcomes of self-assessment available to the Regulators on request. The Chief Assessor's report to the Certification Board will highlight any areas for concern in relation to the Regulatory Arrangements and action will be taken as the highest priority to rectify any shortcomings. 

Return to index