The National Eligibility-cum-Entrance Test (NEET) this year has caused so much chaos and despair that the Supreme Court and the Union Minister have both had to step in. The situation is complex, and so many things seem to have gone wrong that a resolution for a few students may affect thousands of others. Mistakes happen, but accusations of malpractice are not new for India's examination systems. The National Testing Agency (NTA), which conducts NEET and other examinations, certainly has reasons to take a deeper look at its systems and processes.
While many risks cannot be anticipated in mass examinations, when things go wrong - even if accidentally - the accountability lies with the testing authorities. It is their job to deliver assessments that are fair, valid, reliable and timely. That is the golden rule of all assessments.
Solutions Can't Exist In Just Theory
The NTA itself is a young organisation, but it has no dearth of excellent advice. It has venerable advisors and leaders. However, examination operations are a tough arena, and examination operations at scale are even more challenging. Delivering high-quality examination and assessment processes requires at least one of two things: either deep experience that comes from having done it again and again for so many years that all flaws have been refined over time, or a rigorous analytical exercise to anticipate even the most unlikely of risks that can derail not just the examination but also the lives of an entire group of students.
This cannot be just theoretical. The agency must assure itself and its governing authority of flawless and fair examinations by showing evidence of mock tests, surprise checks, managing multiple abilities, dealing with mistakes at centres, and, of course, multiple and almost obsessive scenario-building and testing. Scale-testing is not just about conducting smooth examinations, but also about ensuring that the processes and systems are refined to extremely high quality standards. The NTA now will certainly be telling itself to adopt a quality-first approach, such as, say, Six Sigma methods, in addition to its current systems. Given the situation currently, the agency cannot deny that it has much ground to cover.
NTA Must Be Able To Anticipate Scenarios
In addition to building more practice exercises and a quality focus, there are a few other things that the NTA must reflect upon. The first of these is a set of principles that can guide quick, on-site resolution of problems. If a test paper has started 10 minutes late, the solution must be along the same lines: minutes for minutes. Unless there's a security breach, there is no reason to resolve a problem measured in minutes with grace marks. Similarly, multiple scenarios must be generated and assessed with the help of the invigilation community to develop even more effective guidelines for testing centres.
The challenge of centralisation must be met head-on, where the decisions that are to be made at a testing centre are clear, as are the actions for which the centre head will be held accountable. A delay at a centre cannot be delegated upwards, nor can a mix-up of papers. Other problems, such as the accusation of leaks, are rightly being raised at the most senior levels.
One must recognise that the NTA and its predecessor organisations have wide experience in delivering reliable examinations. At the same time, one must also recognise that though such operational excellence is hard-won, it's not static. It is a constant and dynamic uphill struggle. One cannot rest thinking that a problem or its resolution is final. High-stakes operations need to be improved constantly, or authorities will find themselves facing crises in arenas they thought they had under control.
Tech Remains Underutilised
The NTA brought with it great hopes of next-generation assessments, and with that, a much higher reliance on technology. While progress is being made, there are many other options that technology can enable. One example is individually generated question papers for a fully online centre. Once a question bank is prepared, then question papers can be algorithmically generated for individual students where even if a question is common across papers, it is possibly a different question number. These papers will thus test the same thing, even though they are not identical. Another, even simpler way would include changing the variables in a problem, say, where one student gets a question about 50 widgets, and another works with 30. This way, the methods are the same, but the answers are not - and cheating is, thus, pointless. This also eliminates the possibility of paper leaks.
Every system, especially when new, will have its own challenges and problems. But this is where the agency must learn from its global partners and its own experience to build new checks and balances.
Technology plays another major role in building transparency, and that comes via data analytics. There is already a great deal of data processing involved in exam assessments. A few simple metrics can help weed out problems if made transparent on a timely basis. For example, one of the goals of the NTA is to provide accessible centres. A metric measuring the distance a candidate travelled would provide much-needed transparency, and any centre that has students travelling exceptional distances would stand out as an outlier. Such transparency itself would flag outliers, who would certainly not want to attract attention if their intent is mala fide. Data analytics that asks smart questions can thus act as a shield for the reputation of the agency and protection for the students.
Some other technology solutions may include centralised proctoring on camera in examination rooms across the country, with algorithms marking out unusual patterns to a team of supervisors. While a lot of such mechanisms already exist, adding a layer of verification, such as visibility, can help further. The discussions should span the range of ethics to effectiveness and must be done at the seniormost levels of the assessment community.
Capacity-Building Is Key
Finally, and possibly most fundamentally, building much more capacity in assessments is important. This includes various areas, such as assessment design, scaled examinations, examination operations, testing for skills versus testing for knowledge, standardisation practices using advanced statistics, and much more. India has very few specialists in these areas currently, even though they constitute explicitly stated aims of the NTA.
Even the best of systems recognise and work for continuous improvement to remain the best. However tedious or stressful, high-quality testing is paramount to sustain both the reputation of the system and the futures of lakhs of students.
(Meeta Sengupta is a board member, advisor and mentor on leadership, governance and futures for education and skills.)
Disclaimer: These are the personal opinions of the author