← All components

Mapping

The exploration surface. For the questions you cannot phrase as a query.

Mapping — DataView’s exploration surface

Component documentation · Data Wrangler v4 Written 8 August 2026 against dataview/mapping/page_well_map.py (~8,700 lines) and page_well_documents.py. Behaviour described here has been exercised against the live database.

1 · What Mapping is for

A database of three million wells answers questions you can already phrase. Mapping is for the ones you can’t.

What’s in this area? Who operates it? Which of these wells has a directional survey? What documents do we hold on that lease? Those start as a rectangle drawn on a screen, not as SQL — and the answer is usually another question. The map is where a person goes from “we own data about this region” to a specific well and the document it came from.

It is not a separate application. DataView is the database and the tool for exploring it; Mapping is the second half. The distinction that matters is in section 3: the map draws what the database holds, rather than computing shapes at render time. A line on screen is a row you can query.

2 · What it draws

Layers come from three places, and knowing which is which explains most of what’s confusing about the page.

Stored geography — the dv_* tables

Each of these is a geography column, SRID 4326, written by promote or by a computation that ran once:

That last constraint is not stylistic. A LINESTRING once landed in the survey-footprint column and the whole seismic layer vanished — the page shares one error handler across footprints and lines, so the type mismatch took both. A geometry type in a shared column is a rendering contract, not a detail.

Catalogued geography — straight from the file catalog

Shapefiles classified as FIELD, LAND_TRACT, BOUNDARY or PIPELINE store their outline as WKT directly in GLOBAL_FILE_CATALOG.SPATIAL_OUTLINE, with the category in CATALOG_TABLE. Seismic outlines go to FILE_SEIS_HEADER.SURVEY_OUTLINE. Neither needs a promote — they draw as soon as the file is catalogued.

The WKT is already reprojected to WGS84 and ring-oriented by the extractor, which matters: a clockwise exterior ring read as geography means the whole Earth except this polygon.

Computed at render — seismic coverage

2D surveys are drawn as a deliberately approximate blob — the convex hull of each survey’s line bounding boxes, buffered. Perry’s word for them is “computed blobs”, and the buffer exists so they read as an approximation rather than a claim.

3D surveys prefer the real outline, dissolved per survey, and fall back to a bounding-box rectangle only where no outline exists. That preference took 3D coverage from 3 footprints to 5 and gives truer shapes, since a bbox over-covers a non-rectangular survey.

And H3, for scale

dv_well carries h3_r4–h3_r7. At continental zoom, aggregating 3.9 million wells is a GROUP BY on a string rather than a spatial operation; zooming in is choosing a finer resolution column. It’s also what lets wells from differently-shaped source schemas be counted together — a cell is derivable from a coordinate alone.

3 · Geometry lives in the database

The map issues SELECT geog.STAsText(). It does not compute shapes.

That sounds like an implementation note and it’s the central design position. A well path drawn on screen was computed once — minimum curvature over the survey stations, projected through UTM, stored as a LINESTRING — so the same geometry is available to any query, export or report without re-deriving it, and two people looking at the same well see the same line.

The alternative, computing at render, was built and kept as a second function for a specific case: a live layer that recomputes on every draw, suitable for a field-sized selection where you want the current answer rather than the stored one. Two functions, named for what they do, rather than one that guesses.

Well paths are worth a note of their own, because the computation has three traps that each look like a rendering bug:

Azimuth reference. Survey azimuths are usually true north; adding a north offset to a UTM northing moves along grid north. At 1.3° from a zone’s central meridian that’s 7.4 m over 621 m of displacement. The two implementations of the path calculation disagreed by exactly that until the convergence rotation was added.

Units. A metres survey computed as feet is off by 3.28× and still looks like a well.

TVD versus TVDSS. Fine in plan view, wrong in a cross-section.

A cross-check against each survey’s own reported TVD catches all three before anything is written.

4 · Finding wells

Filters push down to SQL. Operator, county, well type, depth range — applied in the query, not in the browser.

Two things about that were learned the hard way and are worth stating as rules. The dropdown’s options come from a separate query scoped by the spatial clause but never by the attribute clause — options built from the currently-filtered wells collapse to a single value the moment the filter works, with no way back. And every query in the family must agree on the expression: one built COALESCE(w.operator_name, ba.ba_name) against a query that didn’t join ba, which errored, returned a sentinel, emptied the map, and made the control vanish and reset itself.

Drawing a rectangle or circle selects wells into a shadow set, so a well found by drawing stays findable after the fact.

The AI filter takes a natural-language question and returns a JSON spec, not SQL. The spec is applied in pandas over already-loaded wells, and the panel says so. The distinction matters for trust: nothing the model produces reaches the database as a query.

Four things about it are worth knowing because each was a real defect:

Comparisons fall back to text when numeric parsing fails. ISO dates sort correctly as strings, so no date parser is needed — and an unevaluable clause counts as not matching rather than silently passing. Before that, float("2020-01-01") raised inside a bare except: pass and “since 2020” matched every well.

bcp writes an unset value as the literal text NULL, and "NULL" >= "2020-01-01" is true lexicographically. Absent values are now recognised as absent.

The filter was correct and the map ignored it — the drill shadow was merged back in after the filter ran. Now the shadow goes through the same filter.

The panel’s diagnostics render the spec as a readable WHERE clause and apply each clause alone with example values, because a combined zero tells you nothing: sensible filter with no matching data, filter on an empty column, and a model that got it wrong all look identical otherwise.

5 · From a map to an answer

Selecting a well opens the scout ticket — a report assembled from the database: header, tops, surveys, completions, stimulations, DSTs, cores, petrophysics, production.

Beneath it, inside the border so it travels with every export, is the provenance footer:

Then each document as a link with its modified date, and a warning where source ids resolve to nothing. Counts come from introspection — every dv_ table carrying both a uwi and an inventory_id — so a table added to the model appears without an edit. Where a well is entirely bulk-loaded it says so plainly rather than showing an empty box.

That footer is the thing the scout ticket cannot otherwise answer: is any of this traceable, and to what. It never raises; a ticket that failed to render because its footer couldn’t count something would be a poor trade.

From there, Documents and Export are sub-pages reached from the results panel.

6 · Documents on the map

A second page maps wells that have documents, one marker per well with an aggregated count — not one marker per file, since several documents attach to one well.

Position comes from dv_well, not from the document. Perry’s reasoning: multiple documents attach to a well, and dv_well is the consolidated table populated from every source, so the location comes from whichever source carried one. The per-file header remains a fallback, which keeps the original promise that a well appears as soon as it is catalogued.

The accepted trade: a document matched to a well that never promoted won’t appear.

Two joins in that page were quietly wrong and are worth recording as a pattern. The well key had to become COALESCE(GLOBAL_FILE_CATALOG.MATCHED_UWI, header.UWI) — MATCHED_UWI is what the assignment panel and the Excel round-trip write, so every manually assigned UWI was invisible. And the header join had to become a LEFT join, because a file can carry a matched UWI without ever getting a header row.

The companion fix is the one that makes it correct rather than merely fuller: the per-well document list keyed on the header’s UWI, so a well plotted from MATCHED_UWI would have opened an empty document table. Both now share one key expression, defined once precisely so they cannot drift.

The join needs a cast. dv_well.uwi is char(14) and MATCHED_UWI is nvarchar; without CAST(... AS char(14)) SQL Server promotes char to nvarchar and the index goes unusable — the same trap that took a gold join from 3.71s to 0.03s once fixed.

7 · What it refuses to draw

A well with no survey gets no path. Absent geometry is not straight geometry, and an invented curve on a map gets believed.

A well whose path would be a dot doesn’t get a line. Below a minimum closure — about 500 ft — a near-vertical well is a point at map scale, and a drawn line implies a shape nobody can see. Those wells still appear as points. Nothing is lost; a misleading rendering is avoided.

A well with no coordinate never reaches dv_well. That’s promote’s gate rather than the map’s, but it’s why “every well in the database” and “every well the map can draw” are the same set — there is no silent population of invisible rows.

8 · The traps

Streamlit discards a widget’s value when the widget stops rendering. A control gated on a non-empty result loses its state whenever the query returns nothing and reappears at index 0 — which reads as “the filter reset itself” and hides the real failure upstream. Give such controls a fallback so they degrade instead of vanishing.

Action widgets cannot have their value set. Sub-pages keep state alive across a page switch by self-assigning every key — standard, and correct for inputs. It is invalid for buttons, downloads and uploaders, and the error surfaces when the widget is created, on a later run, on a different page. Returning from Export poisoned a button on the map. A name-by-name deny list could not hold; the test is now on how keys are named, and the file is swept in both directions after any new button — 20 action keys with none uncaught, 29 input keys with none wrongly skipped.

Trace outward through every enclosing condition. A geography layer was added correctly and drew nothing, because the block containing it was gated on a different layer being switched on. Same class as a reset button placed inside a collapsed expander that only rendered in one mode. On a page this size, “I put it next to X” is only as good as X’s own visibility.

Expanders do not nest. A block already inside one needs no second disclosure — the parent is the disclosure.

bcp is how the page reads wells, not a different source. The query is dumped to CSV by the bcp utility and parsed positionally against a column list, with no header row — so a mismatch between the SELECT and that list would silently misname every column from the mismatch onward.

Three layers depend on shapely and all degrade silently. 2D zones and geographic entities return nothing; 3D falls back to bounding boxes. Check the library is installed before diagnosing a blank layer as a data problem.

A registry nothing reads is worse than no registry. A layer table survived after its controls were removed, so registering a new layer there did nothing and produced a confidently wrong diagnosis. Dead code that looks live costs more than dead code that looks dead.

9 · Open items

The map can point at a different database from the rest of the app. Its bcp path reads a session value seeded from a module constant rather than from the live connection, so reconnecting mid-session leaves the map pointed elsewhere. The fix is to seed it from the connection.

Cached queries don’t know about a load. Well queries cache for 10 and 60 minutes, and loading or promoting wells doesn’t invalidate them — so the map can serve a stale set for up to ten minutes after a load. The cache-busting parameter exists and is currently a constant; a session version bumped by the loader would close it.

The AI filter’s match count is counted against loaded wells only, so with shadow wells on the map the panel’s “N of M” can read lower than what is drawn. Not wrong, but worth knowing before it’s reported as a new bug.

Two seismic surveys draw in no layer at all — their type is neither 2D nor 3D, and both layers filter on exactly those. Real surveys the classifier didn’t type; they need an unclassified bucket.

2D zones could use the real outlines rather than hulling bounding boxes, which would follow the actual line corridors. Deliberately not done: the hull works, and dissolving hundreds of geometries per render is a real cost. Worth revisiting only if the blobs look too fat.

One MATCHED_UWI per file means a shapefile containing many wells surfaces only its representative well on the documents map, even though capture fanned all of them into dv_well.