Digital twin and multimodal neural networks for automated coastal railway bridge maintenance

Coastal railway bridges are exposed to accelerated deterioration due to harsh marine environments, making their inspection and maintenance both costly and complex. This paper proposes a semi-automated framework that integrates Digital Twin (DT) technology with a Multimodal Neural Network (MNN) to generate natural language repair strategies directly from visual inspection data. The system combines an EfficientNet-based convolutional encoder with a Transformer decoder, trained on a domain-specific dataset of corroded bridge components annotated by experts. Unlike conventional damage detection pipelines, the proposed model outputs actionable, human-readable maintenance recommendations that are programmatically embedded into Industry Foundation Classes (IFC)-based BIM models as structured property sets. This enables seamless integration into Building Information Modelling (BIM)-based DT environments, supporting downstream decision-making and lifecycle asset management. Experimental results show that the model achieves a semantic similarity score of 0.7285 and a BLEU-3 score of 0.4193, indicating strong alignment with expert-authored strategies. While exact match accuracy is limited to 24.18%, this reflects the inherent linguistic variability in valid maintenance descriptions. The system also incorporates expert feedback to support human-in-the-loop learning and continuous improvement. These findings demonstrate the feasibility of combining DL and openBIM standards to enable scalable, automated, and semantically enriched maintenance planning for coastal railway infrastructure.