Can an AI agent build and maintain your data pipelines?

Can an AI agent do data engineering?

Part of it, with a person approving each change. AI agents for data engineering can draft pipeline code, profile and clean raw tables, add data quality checks, write documentation and diagnose failed runs. A data engineer still reviews every change before it reaches production, owns metric definitions and decides what the agent may access.

Two companies describe using one, both quoted by Google. Vodafone's head of engineering says Google's Data Engineering Agent replicated "all existing data and transformations scripts" during its migration to Dataform and cut the time typically required for manual ETL migration by 90%; that is Vodafone's figure, and the method is not published. PRISA Media's lead data engineer says the agent "already delivers value in automating maintenance and small optimizations."

The independent evidence is more cautious. In ELT-Bench, a 2025 research benchmark of 100 pipelines with 835 source tables and 203 data models, the best agent tested loaded data successfully for 57% of pipelines but built only 3.9% of data models correctly, at an average cost of $4.30 per pipeline in model usage. The models tested have since been replaced, so read the gap between loading and modeling as the finding, not the exact numbers.

What the agent does and what stays with you

Agent

1of 10

• Write pipeline and column documentation

Agent with review

7of 10

• Draft pipeline code from a written description of sources, targets and rules
• Profile and clean raw data (types, duplicates, formats)
• Map a source schema to a target schema
• Add data quality checks to tables
• Diagnose failed pipeline runs
• Apply fixes to production pipelines and data
• Translate pipelines during a platform migration

Human

2of 10

• Define business logic and metrics (what counts as revenue or an active customer)
• Grant access, set credentials and decide what the agent may write to

The split follows how easy the work is to check. Compiling code, loading rows and writing column descriptions have a clear right answer. Business logic does not. A join can be valid SQL and still change revenue. DataKitchen's CEO puts the line at intent: whether 222 invalid state codes should become NULL or map to real states "is a business call."

Google's documentation requires you to review and run or schedule pipelines, and Orchestra's template pauses for human approval before applying a proposed fix. Google says its agent can generate a plan listing its objective, assumptions and the files it will modify, and that it "cannot execute pipelines. You must review and run or schedule pipelines." Orchestra's pipeline fixer template has one agent investigate a failed run, posts the error, cause, suggested fix and evidence to Slack for approval, and only then runs a second agent to apply the fix. Not every product works this way: Snowflake's product page for CoCo describes automations that run recurring workflows even when your computer is closed, with an audit trail of each run. The product page does not specify whether these automations have a human-approval gate; confirm the controls available before relying on them.

A worked example: one failed nightly run

This is an illustration, not a customer story. Assume, for this example, that a nightly pipeline builds an orders table and a not-null test on the customer ID column fails for every row in last night's batch.

The agent reads the run logs, queries the source schema with its read-only role, and finds that the source column was renamed. It opens a pull request that maps the new column name to the old one, runs the model in the development schema, and shows a reconciliation query: yesterday's order count and revenue match the source system's totals.

Your engineer reads the pull request in the morning, checks with the source team that the rename is permanent, and merges. If the source team had said the rename would be rolled back, the right answer was to wait, and the agent could not have known that. The agent saved the log reading; the engineer made the call.

What you need to provide

Access is the first decision. Give the agent a read-only warehouse role on source and production schemas, and write access only to a development schema or branch. Changes reach production through pull requests and your normal deployment, never through the agent's own credentials. Orchestra's documentation warns that its second agent "applies changes without further review, so scope the agent's permissions to what it genuinely needs," and advises starting with a single pipeline rather than every pipeline in the account.

The agent will touch whatever data those schemas hold: customer records, orders, often personal information. Google says its agent can read sample rows and data profiles as context, does not use them for training, and can use glossary terms to tag columns containing personal information for special handling. Check your own provider's terms on retention and processing location.

Give it your rules in writing: naming conventions, style guide, metric definitions. Google supports an agent instruction file for exactly this, and notes that agent behavior changes over time with model upgrades, so you should retest your rules. Finally, give it known numbers to reconcile against, such as last month's revenue in the finance report. Data jobs fail silently; a reconciliation check is how you hear about it.

Job brief template

Use this as the brief for a vendor, an operator or your own team. Replace the bracketed values with your own.

The reconciliation criterion does most of the work. Tests catch broken code; only a comparison with a number someone already trusts catches a pipeline that runs and is wrong.

Risks and human checkpoints

For context on the human side of the trade, BLS has no separate occupation for data engineers. The closest it reports, database architects, had a median annual wage of $139,500 in May 2025. This page shows no agent price: Snowflake bills CoCo by token consumption, and no sourced per-pipeline range was found, so there is no cost comparison here.

How to run a trial

Run the agent beside your engineers before you rely on it. Our suggestion: four weeks, long enough to see routine failures and at least one change request.

1. Pick a slice: three to five pipelines with good tests and a known reference number.
2. Set up the access in the brief: read-only on production, write on a development schema, pull requests only.
3. Send it every failed run and change request in the slice. Your engineers keep working as usual.
4. Score each pull request: did it compile, did the reconciliation match, did the reviewer accept it unchanged.
5. Plant a test: introduce a deliberate double-count in a development copy and check whether the agent's reconciliation catches it.
6. Time the reviews. Hours spent checking the agent are the cost you trade against engineering time.

1. 1

   Pick a slice

   Three to five pipelines with good tests and a known reference number

2. 2

   Set up the access in the brief

   Read-only on production, write on a development schema, pull requests only

3. 3

   Send it every failed run and change request

   Your engineers keep working as usual

4. 4

   Score each pull request

   Did it compile, did the reconciliation match, did the reviewer accept it unchanged

5. 5

   Plant a test

   A deliberate double-count in a development copy; does the agent's reconciliation catch it

6. 6

   Time the reviews

   Hours spent checking the agent are the cost you trade against engineering time

If the agent meets the brief's criteria for the whole trial, widen the scope one pipeline at a time. Keep metric definitions, access and production merges with your team.

For structuring the test, see how to run a paid trial with an AI agent and how to supervise an AI agent. Related pages: reporting and BI, spreadsheet analysis, the data analyst role, data entry and the data overview.

Questions buyers ask

Will an agent replace my data engineer? No source here says so. The customer quotes cited describe the agent taking over migration scripts, maintenance and small optimizations. The Google documentation and Orchestra workflow described here keep a human review step; the sources cited do not establish whether every product does. BLS projects employment of database architects, the closest occupation it reports, to grow 9% from 2025 to 2035.

Where should I start? Our recommendation: start with diagnosing failed runs and writing documentation. Both are easy to check, and a wrong answer costs a few minutes of review. Move to pipeline changes once the agent's pull requests pass your reconciliation checks.

Can I let it fix pipelines on its own overnight? Google's agent cannot run pipelines itself, and Orchestra's template waits for approval before applying a fix. Our recommendation: keep that gate. If you remove it, you lose the step that catches a change that runs cleanly and is wrong.

Does it need to see my data? Usually some of it. Google's agent samples rows and profiles to plan transformations. Limit the role to the schemas in scope and keep sensitive columns out where you can.

Sources

1. 1
   Data Engineering Agent overview

   Google CloudPrimaryAccessed Nov 14, 2026

2. 2
   Exploring the Data Engineering Agent in BigQuery

   Google CloudVendorAccessed Nov 14, 2026

3. 3
   ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines

   arXiv (Jin, Zhu and Kang)PrimaryAccessed Nov 14, 2026

4. 4
   Snowflake CoCo: Snowflake-Native AI Coding Agent for Data

   SnowflakeVendorAccessed Nov 14, 2026

5. 5
   Agentic Pipeline Fixer with Human Approvals

   OrchestraPrimaryAccessed Nov 14, 2026

6. 6
   Agents to Fix Data Quality Automatically With Control: A Design Pattern

   DataKitchenVendorAccessed Nov 14, 2026

7. 7
   Database Administrators and Architects, Occupational Outlook Handbook

   U.S. Bureau of Labor StatisticsPrimaryAccessed Nov 14, 2026