A successful go-live can feel like the end of a transformation programme.
The assessment has been designed. The platform has been selected and configured. Processes have been developed. Candidates and users have been prepared. Testing has been completed. And the first live assessment has been delivered.
But go-live changes something important.
For the first time, the whole service is operating for real.
Real candidates are using different devices, networks and environments. Examiners and operational teams are interacting with the platform at scale. Processes that worked in testing are being used under pressure. Technology is interacting with people, policies, communications and other systems in ways that can be difficult to reproduce fully beforehand.
That's when you start learning things that implementation couldn't tell you.
Four weeks after go-live, what do you actually know?
Technology implementations often include a period of ‘hypercare’ after launch.
For four or six weeks, additional project and supplier support remains in place while problems are identified and resolved. At the end of that period, the assumption is that the new service is sufficiently stable to move into business as usual.
For digital assessment, that timetable can make little sense.
Four weeks after your first live assessment, marking and quality assurance may still be underway. Results may not have been published. The main volume of post-results complaints and appeals may therefore still be ahead of you. Candidate and examiner feedback may only just be being analysed.
Most importantly, you may not yet have experienced the complete assessment lifecycle.
And if the assessment only runs a few times each year, four or six weeks of hypercare may tell you very little about whether the service can perform reliably across successive deliveries.
The transition into business as usual should therefore be based on what the organisation has learned, rather than simply how much time has passed.
Have you completed the end-to-end assessment process? Have the critical processes performed as expected? Are recurring issues understood? Have candidate, examiner and operational feedback been reviewed? Are the remaining risks understood and controlled? Is the permanent team ready to take ownership?
That may take longer than the conventional technology implementation model allows.
And it needs to be planned and funded.
Move from implementation to continuous improvement
Once the assessment is live, the challenge changes.
During implementation, the focus is on building and launching the service. Afterwards, the organisation needs a sustainable way of learning from it.
Every delivery creates evidence: candidate and examiner feedback, complaints, support contacts, incidents, operational experience and assessment data. Bringing that evidence together helps distinguish individual frustrations from recurring problems and identify where improvement will make the greatest difference.
But not everything can be fixed at once — and not every problem needs a technology solution.
Assessment integrity and fairness come first, with particular attention to issues affecting candidates during live delivery where there may be little opportunity to control the impact. Beyond that, Candidate Experience, operational risk and efficiency all need to inform the priorities.
And priority doesn't necessarily mean immediate release.
A significant issue may be safely controlled through an operational workaround while a permanent solution is properly designed, tested and introduced at an appropriate point in the assessment cycle.
Continuous improvement therefore requires judgement: understanding the problem, deciding what matters most and making the right changes at the right time without introducing unnecessary risk.
Become an intelligent customer
As the implementation project recedes, responsibility increasingly moves to the permanent organisation.
That transition matters particularly when the assessment depends on an external technology provider.
The supplier understands its platform. The assessment organisation needs to understand what that platform must do for the assessment — and how to know whether it is doing it safely and reliably.
That means retaining enough knowledge and capability to be an intelligent customer.
The organisation needs to understand its critical requirements and dependencies, challenge proposed solutions, scrutinise incidents and root causes, participate properly in testing and releases, and ensure that technical decisions continue to take account of assessment integrity and Candidate Experience.
It also needs an effective relationship with the supplier after implementation.
Development capacity, commercial arrangements and post-launch support should be understood before go-live. So should the way priorities will be agreed when the organisation's assessment needs have to compete with the supplier's wider product roadmap.
A successful procurement doesn't end with selecting the right platform.
And the same principle applies to future development.
The roadmap shouldn't belong to the technology supplier. It should belong to the assessment service.
Build an operating model that can handle the exceptional
As the assessment matures, most problems should be managed through normal operational processes.
But a mature operating model also needs to be ready for the occasions when something bigger happens.
A significant assessment problem can quickly extend beyond the exams team. It may require technology expertise, communications, customer or membership services, senior decision-making, supplier management, regulatory engagement or other parts of the organisation.
Trying to create that structure while an incident is unfolding wastes valuable time.
Organisations should know when an operational issue becomes an organisational one, who has the authority to make that decision and how the right people will be brought together quickly.
That might mean activating a predefined incident team or task force, with clear leadership and individual workstreams for the assessment, technology, communications and other consequences.
Importantly, the purpose isn't to bring more people into managing the assessment itself.
It's often the opposite.
The wider organisation can manage stakeholder communications, enquiries, escalation and coordination while the people with the deepest assessment knowledge concentrate on understanding what has happened, protecting candidates and maintaining the integrity of the assessment.
Resilience, therefore, isn't purely technical.
It's the ability to prevent what you can, detect what you can't, and respond effectively when something does go wrong.
Stable doesn't mean finished
The objective isn't to keep an assessment in a permanent state of change.
As the service matures, recurring problems should reduce, processes should become more predictable and development should become increasingly considered rather than reactive.
But business as usual shouldn't mean that learning stops.
A mature assessment service continues to learn from Candidate Experience, operational evidence, assessment data and the people delivering it — using that evidence to make considered improvements across technology, processes, preparation, support and the assessment itself.
A successful go-live gets an organisation into live delivery. A mature operating model ensures the service continues to learn and improve once it gets there.
Go-live isn't the end of transformation. It's the beginning of running, learning from and improving the service.