Data Engineer

Moving data is easy.
Stopping bad data is the job.

Two people asking the same question should get the same number. Our data engineers build the tests, contracts and gates that make that true — and stay on the Virtus payroll while they sit in your team.

An operator watching data dashboards
$35–62
per hour
7–12 days
to day one
1 month
minimum
The DAG, with gates

Six nodes, six gates, and what happens when each one fails

Every pipeline diagram you have been shown has the boxes. Almost none of them have the third column — and the third column is the entire difference between a pipeline and a liability.

extract_orders
Gate

Source row count within 3σ of the trailing 30 days

On failure

Pipeline halts. Yesterday's data stays served. Nobody sees a half-loaded table.

extract_crm
Gate

Schema hash matches the contract

On failure

Quarantine the batch and page the owner. A renamed column is not silently mapped to null.

stage_union
Gate

Referential integrity: every order has a customer

On failure

Orphans land in a rejects table with the reason, and the run continues without them.

model_revenue
Gate

Uniqueness on the grain, no unexpected nulls

On failure

Build fails before publish. The old model stays live — dashboards never show a duplicated total.

publish_semantic
Gate

Reconciles to the finance close within 0.5%

On failure

Publish blocked and the variance report is sent to both owners. This is the gate that keeps the metric trusted.

freshness_probe
Gate

Served data less than 90 minutes old

On failure

A staleness banner appears on the dashboard itself, so consumers know before they act.

What we interview for

Four opinions, held firmly

If you need the answers, not the pipes

A dashboard nobody trusts is worth nothing

The cost of a wrong number is not the wrong number — it is that every future number gets checked by hand. Gates are how trust is rebuilt.

Tests on data, not only on code

Your transformation code can be perfectly correct and still produce nonsense, because the source changed. Only data tests catch that.

Cost is a design constraint

Warehouse spend attributed per model and per consumer, with the ten most expensive queries named and owned.

Lineage that is generated

Column-level, from source system to the number on the slide, produced by the tooling rather than drawn in a diagram that goes stale.

The stack

Warehouse-agnostic

Orchestration
Airflow · Dagster · Azure Data Factory
Transformation
dbt · Spark · Polars · plain SQL
Warehouses
Snowflake · BigQuery · Databricks · Redshift · Synapse
Streaming
Kafka · Kinesis · Debezium CDC
Quality & catalogue
dbt tests · Great Expectations · Monte Carlo · DataHub
Seniority

Three bands

Mid-level

3–5 yrs$35–43/hr

Builds and maintains models in an existing warehouse. Comfortable with dbt and SQL; still learning orchestration failure modes.

Senior

5–8 yrs$43–54/hr

Owns a domain end to end — ingestion, model, tests, cost and the on-call for it.

Lead

8+ yrs$54–72/hr

Sets the platform, the contracts with source systems, and the semantic layer everyone else builds on.

Name the number two teams disagree about

That single disagreement tells us more about what you need than a stack list. It is usually where the first engagement starts.