Since its inception in the 1950s, the mainframe has remained one of the most popular systems in the enterprise. Even today, over 70% of Fortune 500 companies and 50% of Global 2000 companies run one or more mainframe systems and continue to maintain COBOL and DB2, despite having many other more modern options in the cloud.

Despite its popularity, the mainframe remains one of the toughest areas for data governance teams, and most of it is due to traditional data lineage challenges as well as modern AI not handling COBOL at the required accuracy and scale. Ironically, that is usually where regulators, auditors, and modernization programs demand the most precise answers about where data comes from and how it was transformed.

This article is a technical deep dive into how Foundational solves that problem — by parsing and analyzing the mainframe source code itself. We'll walk through real COBOL, JCL, copybooks, and DB2 DDL from AWS CardDemo, an open-source mainframe credit-card application, and show how static analysis turns them into field-level lineage.

Ask where the world's money actually moves and the answer is still, to a remarkable degree, an IBM Z machine running z/OS — IBM's mainframe hardware line and its operating system. Banks, insurers, card networks, government agencies, airlines, and large retailers keep their systems of record on the mainframe because it delivers what nothing else has matched at that scale: decades of accumulated, battle-tested business logic running with extreme reliability and throughput. Most of that logic is written in COBOL, a programming language created in 1959 specifically for business data processing. It remains the workhorse of banking and insurance back ends: verbose by modern standards, but purpose-built for one job — reading fixed-format records, applying business rules, and writing fixed-format records, millions of times a night.

That environment is far more than "COBOL code". A typical z/OS estate combines several kinds of assets, each with a rough equivalent in the modern stack:

Data constantly crosses the boundaries between these components: a DB2 table is unloaded to a flat file, a COBOL batch job reads that file and rewrites a VSAM master, an extract lands somewhere a Python job picks it up. Understanding how a single field travels through all of that is the problem this article is about.

Take a concrete question a bank's data team might face: "Our account balances feed the regulatory capital calculation. What exactly updates ACCT-CURR-BAL in the account master, and from which inputs?"

On the mainframe, the honest answer spans several assets. In CardDemo, the nightly posting job POSTTRAN runs a COBOL program called CBTRN02C, which posts the day's card transactions to the master files:

Reading left to right: the day's transactions arrive as a plain sequential file, DALYTRAN.PS. For each transaction, CBTRN02C looks up the card number in the cross-reference file CARDXREF to find the owning account, then validates the transaction against the account master ACCTDATA.

Valid transactions are appended to the posted-transactions file TRANSACT, the account's balances in ACCTDATA are updated in place, and running per-category totals are updated in TCATBALF. Anything that fails validation lands in the day's reject file, DALYREJS.

So a single transaction amount arrives in a flat file, passes through validation logic that reads two other VSAM files, and ends up — transformed differently each time — in three separate datasets. None of this is a SQL query. All of it is field-level data flow that someone, eventually, will need to explain to an auditor or untangle for a modernization project.

Three situations force organizations to answer this kind of question precisely:

Regulatory provenance — Frameworks like BCBS 239 — the Basel Committee's principles for risk-data aggregation and reporting — don't use the word "lineage", but in practice supervisors and auditors expect banks to demonstrate, down to the critical data element, where reported numbers come from and what transformed them. A vague answer that points at the mainframe won't satisfy them; "this balance is the accumulation of validated transaction amounts from the daily feed, posted by this program, gated by this credit-limit check" will.

Modernization — Every mainframe modernization program — rewriting a component in Java, moving a workload to the cloud, retiring a feed — starts with the same question: what breaks if we change this? Without lineage, teams reverse-engineer data flow by reading code and interviewing whoever is left who remembers. That's slow enough to stall projects.

Understanding how a field is actually used and calculated — Before changing the semantics of a field (say, making customer currency multi-valued), you need every downstream consumer: which programs read it, what they compute from it, and which reports it lands in. On mainframes this includes consumers connected only through intermediate files, which no catalog records.

Most data lineage products are built on two sources of truth: SQL query history and warehouse metadata. Both go dark at the mainframe boundary, for structural reasons:

Most mainframe data movement isn't SQL — The CardDemo posting flow above touches six datasets and issues zero SQL statements — it's all COBOL READ/WRITE/REWRITE against VSAM and sequential files. A tool that reconstructs lineage from query logs has literally nothing to parse. This isn't an edge case; file-based batch is the backbone of mainframe processing.

Even the SQL paths lose track of where the data lands — A SELECT names its sources, never its destination — and on the mainframe, the destination usually lives outside the database entirely, in JCL. Consider CardDemo's reference-data extract job TRANEXTR, which unloads a DB2 table to a flat file using DSNTIAUL, IBM's sample unload utility. There is no COBOL involved at all — the SQL lives directly inside the JCL. Two pieces of plumbing make the excerpt below readable: TSO (Time Sharing Option) is z/OS's command environment — in effect, the mainframe's shell — and IKJEFT01 is the standard program for running TSO commands inside a batch job. The SYSTSIN stream near the bottom holds those TSO commands, and it is what actually launches DSNTIAUL. DSNTIAUL in turn reads the SQL to run from the SYSIN stream (excerpt; line numbers added for reference):

To read the code snippet above: line 1 runs IKJEFT01. Lines 2-3 (SYSREC00) allocate a brand-new dataset, TRANTYPE.PS — the unload target. Line 4 (//SYSIN DD *) opens an in-stream input block: lines 5-12 are the SQL statement itself, embedded in the job, and the /* on line 13 terminates the block. Lines 14-19 (SYSTSIN) hold the TSO commands: connect to the DB2 subsystem DAZ1 (a subsystem is a named DB2 instance), then run DSNTIAUL (the trailing - on lines 16-17 is just line continuation). The layout looks inside-out — the SQL on lines 5-12 sits above the command on line 16 that runs it — but that's normal JCL: DD statements are declarations, not sequential steps, so their order within the step doesn't matter.

Now the lineage aspect: this SELECT does show up in the query history — DB2 will record that the statement ran, and even which job ran it. What the database cannot record is where the result set went: the sink is the SYSREC00 DD statement on line 2, a dataset allocation that exists only in JCL. From DB2's perspective, the rows simply left. The file TRANTYPE.PS then feeds downstream report programs, and that entire onward chain is invisible to any database-side monitoring.

Foundational on CardDemo: the reference-data flow from the excerpt above — the DB2 table TRANSACTION_TYPE (subsystem DAZ1, schema CARDDEMO) with TR_TYPE selected, flowing into the flat file TRANTYPE.PS. Foundational also shows the relevant code snippets from the JCL file.

Query history shows final values, not logic. Static SQL embedded in COBOL is precompiled and bound into packages; what executes at runtime carries host variables (:INPUT-REC-DESC), not the COBOL logic that produced their values. Even with full statement-level capture, you'd see that a column was set — never why, or from which upstream fields the value was computed.

And these approaches don't just fall short — they demand risky access to get even that far. Everything above shows that watching the database cannot reconstruct mainframe lineage. Yet to deliver even its partial picture, a database-side tool must connect to the production DB2 that holds core banking data, read its catalog metadata, and run queries or monitoring traces against live systems. Those are the most tightly guarded systems in the enterprise, and putting third-party workload on them is exactly the kind of operational risk mainframe security teams exist to prevent — so that access is almost never granted. An approach that is blind to file-based data flow and needs production database access to show anything at all is, for most mainframe estates, dead on arrival.

If the logic lives in code, the code is where lineage should come from. Static analysis — parsing source into structured form and tracing data flow through it — has properties that runtime observation fundamentally lacks:

Foundational's architecture deliberately mirrors a compiler toolchain, because the problem has the same shape: many separately-owned compilation units that must be linked into one consistent whole.

This is the same architecture that lets conventional compilers scale to codebases of arbitrary size, and it works for the same reason: each unit is analyzed once, deeply, in isolation — and global consistency is a separate, well-defined resolution phase.

Let's walk through the actual CardDemo posting program, CBTRN02C.cbl.

Step 1: Parse to an AST. Enterprise COBOL is not a trivial parse. Source lines are fixed-format: each of the 80 character columns of a line has an assigned meaning — columns 1-6 hold a sequence number, column 7 is an indicator (an * there turns the whole line into a comment), the code itself lives in columns 8-72, and columns 73-80 are reserved for identification. On top of that come COPY members that must be expanded before the program even makes sense, REDEFINES overlays, reference modification (COBOL's substring slicing), and EXEC SQL blocks that are a different language entirely — handled on real systems by the DB2 precompiler or the integrated SQL coprocessor.

Below is the paragraph of CBTRN02C that updates the account balances — in COBOL, a "paragraph" is a named block of statements, roughly a small function (condensed: file-status checks and error handling elided):

Simplified, the AST for that paragraph looks like:

Step 2: Resolve symbols through copybooks. DALYTRAN-AMT isn't declared in the program — it comes from copybook CVTRA06Y, pulled in with COPY CVTRA06Y.:

The copybook is the schema. It tells the analyzer that DALYTRAN-AMT is a signed amount with nine integer and two implied decimal digits — 11 digits of zoned decimal, with no physical decimal point in the record. And because the same copybook is used by every program that touches this file, it is also what lets the linker later recognize that two programs are reading and writing the same record structure. Likewise, ACCOUNT-RECORD comes from copybook CVACT01Y and describes the 300-byte account master record.

Step 3: Trace read-sources to write-sinks. The analyzer walks the AST and connects the dots: DALYTRAN-RECORD is populated by READ DALYTRAN-FILE INTO DALYTRAN-RECORD; ACCOUNT-RECORD is written back by the REWRITE ... FROM ACCOUNT-RECORD. Along each path from a read to a write, it collects every assignment, arithmetic statement, and governing condition, and composes them into the end-to-end transformation logic for that source-to-sink pair — so the ADD and the IF from the AST above become a single lineage edge from DALYTRAN-AMT to ACCT-CURR-CYC-CREDIT that carries both the arithmetic and the condition under which it applies. For this one program the extracted field-level lineage includes:

Two details here matter to anyone who has tried to do this with regexes. First, the processing timestamp flows through a REDEFINES overlay: FUNCTION CURRENT-DATE is split into components and reassembled into DB2 timestamp format through a redefined byte layout. Tracking that requires modeling COBOL storage semantics, not text.

Second, the credit-limit check — COMPUTE WS-TEMP-BAL = ACCT-CURR-CYC-CREDIT - ACCT-CURR-CYC-DEBIT + DALYTRAN-AMT, then compared against ACCT-CREDIT-LIMIT — is a control dependency. It decides whether the transaction posts but contributes no bytes to the posted records; its outcome only selects the reject-reason constant written to the reject file. That is exactly why data flow and control flow must be modeled separately: a sound analyzer keeps the two distinct, since conflating them floods the graph with edges that aren't real provenance.

Step 4: Resolve logical files to physical datasets via JCL. Inside the program, files have logical names: SELECT DALYTRAN-FILE ASSIGN TO DALYTRAN. The name DALYTRAN is a DD name — a symbol resolved from the JCL when the job step runs. The POSTTRAN JCL provides the binding (excerpt):

Only by joining the COBOL analysis with the JCL analysis does the lineage arrive at real, catalog-level dataset names — including normalizing GDG generation syntax like DALYREJS(+1) so that every day's reject file is recognized as the same logical asset. The two views also cross-check each other: CBTRN02C's reject record is 430 bytes (a 350-byte transaction image plus an 80-byte validation trailer), matching the DD's DCB=(RECFM=F,LRECL=430,...) exactly. The same JCL analysis covers utility steps: an IEBGENER step copying SYSUT1 to SYSUT2, or a DSNTIAUL unload writing to SYSREC00, produces lineage edges with no COBOL involved at all.

Foundational on CardDemo: the account master ACCTDATA.VSAM.KSDS with ACCT-CURR-BAL selected and its upstream feed DALYTRAN.PS in the lineage graph; the panel on the right shows the exact COBOL statements behind the edge, from CBACT04C.cbl and CBTRN02C.cbl, linked to the source on GitHub.

Mainframe technologies aren't commonly placed in a single repository. DDL lives with the database team, JCL sometimes lives with operations, COBOL lives in one or many application repos, and the modern Java/Python services that consume mainframe extracts live somewhere else entirely. Each repository is analyzed independently — so each snapshot contains references it cannot resolve alone. The linker's job is to resolve them. Here are three examples of what that takes:

1. Star expansion across repositories. Suppose the reference-data unload used a star query — a pattern we see constantly in customer estates:

The repository containing this unload job doesn't define the table, so the "compiler" phase can only emit a partial edge: CARDDEMO.TRANSACTION_TYPE.* → TRANTYPE.PS, flagged as unresolved — the analyzer knows data flows, but not which columns. A different repository, owned by the database team, holds the DDL:

The linker joins the two snapshots: it resolves CARDDEMO.TRANSACTION_TYPE to its schema, expands the * into the explicit column list TR_TYPE, TR_DESCRIPTION, and replaces the star edge with two precise column-level edges into the flat file's record layout. Exactly like a linker patching an unresolved symbol with the address from another object file.

2. Qualified-name inference. A COBOL program refers to the table as CARDDEMO.TRANSACTION_TYPE — schema and table, with the database location implied by the runtime environment. A Java service reading the same table over JDBC might reference it through a connection string plus an unqualified name, or a three-part name. If the linker naively treats each spelling as a distinct entity, the lineage graph splits into disconnected islands at exactly the boundary customers care most about — where the mainframe meets modern code. Foundational's linker normalizes and unifies these identities across snapshots so a single table is a single node, no matter which side referenced it.

3. External vs. internal names. Suppose a Java service calls a DB2 stored procedure — CALL CARDDEMO.CALC_INTEREST(?, ?, ?). The caller's snapshot knows only positional arguments. The procedure's own definition, in another repository, declares named parameters and contains the actual logic. The linker matches call sites to definitions and binds arguments to parameters by ordinal position, so lineage flows through the procedure: from the Java variable, into the parameter, through the procedure body, into the table column.

None of the above is a weekend project, and it's worth being honest about why:

Mainframes hold the most regulated, highest-stakes data in the enterprise, and its lineage runs through COBOL logic, copybooks, JCL, flat files, and DB2 — mostly outside SQL, and therefore mostly invisible to query-history-based lineage tools. Static code analysis is the right foundation: it covers every kind of data movement, explains the transformation logic rather than just observing outcomes, sees every possible path, and requires access to source code only — not to production systems. Foundational applies a compiler/linker architecture to this problem: per-repository analyzers extract deep, field-level lineage snapshots, and a linking phase stitches them into one consistent graph across mainframe and modern platforms alike — the same architecture that makes ordinary software builds scale.

If you're planning a modernization program, facing a regulatory lineage requirement, or simply want to know what actually feeds a field in your account master — we'd like to show you Foundational running on your own codebase. Contact us for a demo.