AI/ML,Full-Stack,Mobile,Research

SURE Safety

AI-powered safety reporting for offshore energy. Workers photograph a hazard; a graph of agents drafts the structured observation, and the worker corrects it or throws it away.

FlutterFirebaseGPT-4o VisionRAGPineconeFastAPI

Offshore workers report hazards by photographing them. What arrives at the other end is not a caption but a structured safety observation, drafted by a graph of agents and handed to the worker to correct or throw away.

The agent graph

inputPhotograph

A worker snaps a potential safety concern. Location and timestamp travel with the image; drafts persist on the device and sync when a connection returns.

image

orchestratorDescribe the scene

Reads the photograph and writes down what is there the way a safety engineer would describe it. Every agent after this one reasons over that written scene, not over the pixels, so there is a single artefact to inspect when a report comes out wrong.

scene description, to all three

agent 1Energy inventory

Haddon · DOE-HDBK-1100 · ETBA

Walks all seven energy categories and marks each present or absent with a confidence, flagging whether it was seen or only inferred from the equipment and the work.

agent 2Barrier assessment

Swiss cheese · bow-tie · LOPA

Checks what should stand between each energy source and a person: isolation, bleed-down, permits, gas testing, guards, PPE, relief devices, fall and fire protection.

agent 3Failure projection

Recognition-primed decision · HAZOP

Runs the accident forward: initiating event, release mechanism, exposure pathway, consequence, severity, then the second-order chain and HAZOP deviations.

three findings

hubAggregator

Merges the three readings and decides what runs next. Every agent returns here rather than calling the next one, so the control flow stays in a single place as agents are added.

merged findings

retrievalFetch prior investigations

Queries prior offshore incident investigations against the merged findings rather than against the photo, which is a far better query than anything derivable from the image alone.

graderRelevance checkretry up to 3x

Corrective RAG. Retrieved documents are graded and the query reformulated up to three times. If nothing relevant surfaces, the report is written without retrieval rather than grounded in loosely related precedent.

findings + documents, or findings alone

writerReport writer

Composes the structured observation from all three analyses and whatever retrieval survived the grader: title, description, concern type, severity, contributing causes and corrective actions.

draft

gateThe worker decides

Every generated field arrives flagged and tap-to-edit. The worker confirms, rewrites or deletes any of it, adds what a photograph cannot show, and chooses whether the observation is filed at all. Nothing reaches the record automatically.

The agent graph behind a single photograph, from capture to the human decision that ends it.

Pixels become prose before anything reasons over them. The orchestrator writes the scene the way a safety engineer would describe it, and every agent works from that text. When a report comes out wrong there is one artefact to inspect.
Three agents, three doctrines, run in parallel on the same description. Each encodes an established safety-engineering method rather than a general instruction to look for hazards: an energy inventory drawn from Haddon's energy model and DOE-HDBK-1100, a barrier assessment drawn from the Swiss cheese model, bow-tie and LOPA, and a failure projection drawn from recognition-primed decision making and HAZOP.
The methods make the agents catch what a camera cannot show. The energy agent evaluates all seven categories and must consider pressure wherever there is process equipment, whether or not a gauge is in frame. The barrier agent treats a control it cannot see as a finding rather than a pass. The projection agent runs the accident forward to a consequence and a severity, then asks what the second-order chain does.
The aggregator is the hub, not a step. Agents never call each other. Each returns to the aggregator, which merges the findings and decides what runs next, so the control flow stays in one place as agents are added.
Retrieval runs on conclusions, not on the photo. Prior offshore incident investigations are fetched against the merged findings, a far better query than anything derivable from the image alone.
Corrective RAG, with a floor. Retrieved documents are graded for relevance and the query is retried up to three times. If nothing relevant surfaces, the report is written without retrieval rather than grounded in loosely related precedent.
The worker decides. Every generated field is flagged and tap-to-edit. The worker confirms, rewrites, deletes, adds what a photograph cannot show, and chooses whether to file the observation at all. The system drafts; it does not determine.

Built and delivered

Flutter app for iOS and Android, with offline drafts that sync when a connection returns. Offshore connectivity cannot be assumed.
FastAPI service orchestrating the agent graph, with Pinecone behind retrieval over indexed BSEE offshore incident investigations.
Firebase for the record: photo, location, timestamp and reporter, with visibility scoped by role and status tracked from Reported through Closed.
First-author paper accepted at the IISE Annual Conference 2026, funded by the NASEM Gulf Research Program. I designed the architecture, built the app and wrote the paper.