AI for Legacy Modernisation — Beyond the Code Translation Demos

AI-assisted legacy modernisation is being pitched at every enterprise we work with that has a meaningful legacy estate. The pitches are compelling: AI translates COBOL to Java, AI extracts business logic from mainframe code, AI generates documentation for systems that have none, AI accelerates migration projects from years to months.

The reality, from our engagements with several large modernisation programmes that have AI assistance built in, is more nuanced. AI does accelerate parts of modernisation that were never the bottleneck. The hard parts remain hard. The migration teams that ship are the ones that have a clear-eyed view of which part is which.

What AI actually accelerates

Where the productivity gains are real:

Code understanding

Reading and summarising legacy code is the first place AI helps. For developers unfamiliar with COBOL, PL/I, or older patterns of dialect-specific code, an AI assistant that can explain what a program does is meaningful productivity.

The pattern: paste a procedure; ask the assistant to explain it; verify the explanation against the team's understanding. The assistant is a fast first read; the developer remains the judgment layer.

Documentation generation

Drafting documentation for code that has none. The assistant produces a reasonable first draft; the documentation team curates. The result is documentation that exists, which is better than the alternative.

For systems with hundreds of thousands of lines of undocumented code, generated documentation that is 70% accurate is a real starting point. Improvements come from the team's review.

Test generation

Generating tests for code that has no tests is one of the highest-leverage AI use cases in legacy modernisation. Each generated test is a candidate; the team curates. Coverage that would have taken years of effort accumulates in months.

The tests aren't always right; the team has to review and adjust. But the candidate generation is the bottleneck; once that's accelerated, the curation is faster.

Code translation for narrow patterns

Translating specific, well-defined patterns from one language to another. A particular COBOL paragraph that does a specific calculation; translate to Java equivalent. The translation is reviewed and tested.

This works for narrow patterns. It does not work for whole-program translation; see below.

Data mapping

Mapping data structures between source and target systems. An ID format change, a column rename, a unit conversion. The assistant produces a candidate mapping; the team verifies.

What AI doesn't accelerate

The parts that remain bottlenecks despite AI assistance:

Understanding the business

Legacy systems encode business logic accumulated over decades. Much of that logic is in the code; some of it is in operational procedures, custom configurations, manual overrides, and tribal knowledge. The system does what it does because of choices made years ago by people who are no longer with the organisation.

AI cannot recover this context from the code alone. The team has to do the business work — talking to users, examining operational procedures, interviewing veterans, comparing historical records. This is the largest single bottleneck in most modernisation programmes; it is unchanged by AI.

Architecting the target

Deciding what the modernised system should look like is judgment work. The new architecture has to fit the organisation's current capabilities, future direction, and operational constraints. It has to account for systems that surround the legacy (integration partners, downstream consumers, upstream sources). It has to satisfy regulatory and compliance requirements that may have evolved.

AI helps with point decisions (this pattern or that pattern) but cannot replace the architectural judgment for the whole programme.

Migrating data

Data migration is rarely the easy part of modernisation. Source data has quality issues; the target schema has constraints; the cutover has timing requirements; the parallel-run reconciliation has to be designed. Each of these is engineering work that doesn't speed up with AI.

AI helps at the margin (suggesting mappings, identifying potential data issues, drafting validation queries). The substance of the migration remains.

Managing organisational change

The new system changes how people work. Workflows change. Roles change. Reports change. The change management is half the programme's work; AI doesn't touch it.

Achieving production readiness

A migrated system has to meet the same production bar as the legacy system it replaces. Reliability, performance, security, observability, operational support. The engineering work to reach the bar isn't accelerated by AI; it's accelerated by good engineering discipline.

Cutover planning

The actual transition from old to new is a complex programme of work. Parallel running, reconciliation, fallback procedures, communication, training. AI doesn't accelerate any of this.

Where the productivity claims overpromise

Some specific claims that don't hold up:

"AI rewrites COBOL to Java"

The pattern works for paragraph-level translation, fails at program-level. Programs have structure beyond the syntactic elements; that structure has to be designed deliberately in the target. Naive translation produces Java that looks like COBOL written in Java syntax — neither idiomatic nor maintainable.

The successful pattern: AI for paragraph-level assistance; human architects for program structure; deliberate target design.

"AI extracts business logic for refactoring"

Business logic is in the code only partially. Configuration, runtime data, operational procedures all encode logic. AI surfaces what is in the code; the team still has to find the rest.

"AI reduces modernisation projects from years to months"

The headline reduction is generally overstated. The parts AI accelerates are often the parts that weren't the bottleneck. The parts that were the bottleneck — business understanding, data migration, organisational change, cutover — are largely unchanged.

A well-run AI-assisted modernisation can be 20-40% faster than its non-AI equivalent. A poorly-run one isn't faster at all because the team didn't put effort into the parts AI doesn't help with.

The realistic pattern

A modernisation programme that uses AI well:

Phase 1 — discovery

The team understands the legacy system. AI helps read and document code. Business analysts work with users, operational staff, veterans. The output is a clear picture of what the system does and why.

AI is a productivity multiplier here, but the team's understanding is the deliverable.

Phase 2 — architecture

Architects design the target. AI helps with specific decisions; the architects own the overall design. The output is a target architecture that fits the organisation.

AI's contribution is at the margins; the architecture is human work.

Phase 3 — incremental migration

The migration proceeds in slices. AI assists with code translation, test generation, data mapping. The team verifies each slice. The output is a working migrated system.

AI productivity gains are largest in this phase. They show up as faster slice completion, more test coverage, better documentation.

Phase 4 — cutover and stabilisation

The transition happens. AI is largely irrelevant; the work is operational. The team manages parallel running, reconciliation, fallback, user adoption.

Phase 5 — decommissioning

The legacy system is retired. The work is administrative and procedural; AI doesn't matter.

The pattern that works: AI is a strong productivity tool in phases 1 and 3. It's a marginal contributor in phases 2 and 5. It's irrelevant in phase 4. Plan accordingly.

What we keep seeing

Recurring patterns in AI-assisted modernisation engagements:

Teams that buy the headline numbers are disappointed. The 10x productivity claims don't materialise. The 1.5-2x gains are real and meaningful but not as exciting.

The bottlenecks shift. When AI accelerates code understanding, the bottleneck moves to business understanding. When AI accelerates testing, the bottleneck moves to data migration. The total time depends on the slowest remaining piece.

The discipline of legacy modernisation is unchanged. AI is a productivity tool; the discipline that produces successful modernisation is the same as it has been for decades.

Teams without modernisation experience expect AI to fill the gap. They don't. Modernisation skill is required regardless.

Tooling integration matters. AI tools that integrate with the team's existing modernisation tooling produce value. Tools that are separate surfaces don't get adopted.

What we recommend

For enterprise teams considering AI-assisted modernisation in 2024:

1. Use AI as a productivity tool, not as a strategy. The strategy is conventional modernisation; AI accelerates parts of it.
2. Identify which parts are the actual bottlenecks. AI helps where the bottleneck overlaps with code understanding and generation. Other bottlenecks need other treatments.
3. Maintain modernisation discipline. AI does not substitute for the engineering practices that produce successful migrations.
4. Plan business-understanding work explicitly. It is the largest single bottleneck and AI does not help.
5. Stage the AI adoption. Phase 1 and 3 are the highest-leverage; other phases benefit less.
6. Measure realistically. 20-40% acceleration is a real and valuable outcome; chasing 10x sets up failure.
7. Engage modernisation specialists, not AI specialists. The work is modernisation; AI is a tool within it.

AI is a meaningful contributor to legacy modernisation in 2024. It is not a substitute for the engineering, business, and change management work that modernisation requires. The teams that use AI as one tool in a disciplined programme ship faster and with higher quality. The teams that treat AI as a strategy ship the same problems they would have shipped without it.