The image of health. Overcoming obstacles in AI-based container damage detections

by Kishor Arumilli, Vice President – Engineering, ATAI

Freight containers, essential to global trade, are prone to various damages due to harsh environments, frequent handling, and constant physical stress. Assessing the condition of these containers from a damage perspective – also known as their health – is crucial for all entities involved in the logistics supply chain. Typically, container terminal operators conduct damage surveys for all incoming (and sometimes outgoing) containers, a procedure that is time-consuming and also has direct implications for personnel safety. As a result, automated container damage surveys are increasingly seen as a promising solution to this issue.

Container damage is any kind of structural deformity that may reduce the container’s value. The ISO 9897 standards codify these damages, enabling various stakeholders to effectively communicate the damage type, location, and severity.

However, the relevance, significance, and impact of the damage vary among stakeholders, resulting in differing purposes for container damage surveys. Each container terminal has its own criteria/ rules for what they consider as damage to the container (usually, a deep sea port terminal may overlook surface dents and rust patches). Additionally, the handling procedures for the damaged container vary from terminal to terminal. This variability in implications and procedures regarding what container damage means presents the real challenge.

The why’s and how’s of surveying container damage

Damage to the container may lead to the damage of freight inside and also impair the container’s utility. In the process of transporting cargo from seller to buyer, the box undergoes handling by various entities and is susceptible to damage at each stage. As per the current practice, the liability is with the entity responsible for the damage. Keeping this in view, a terminal conducts the container damage survey when it enters the terminal (entry at the terminal gate, container unloading from the vessel or rail). As a standard practice, clear pictures of the damaged container are taken as evidence, and a sign-off is obtained from the relevant entities.

Besides protecting the terminals against liability claims, the container damage survey also helps in making operational decisions. Specific damages make it unsuitable for using certain container handling equipment. For example, damage to the corner cast may make a container unsuitable for spreader-based handling, but fork-based handlers can still move such boxes. Based on the cargo type and the damage assessment, the terminal may, too, change the container storage location in the yard. This also leads to a change in procedures for the downstream activities related to that container. These operational decisions hinge on the container damage surveys performed for the incoming boxes. Initially, damage surveys were conducted manually by surveyors, who physically inspected the container from all visible sides and documented the information on paper. Additionally, pictures of the damage were taken and stored for further reference. An improvement to this method involved providing a live visual feed of the container from all visible sides to the operator sitting at a workstation. Because of the constrained space environment at the terminals, the full container picture was constructed using line scanners or image stitching techniques. Computer vision and deep learning have significantly helped automate container damage detections. A stitched image (from the line scanners) of the incoming container is provided as input to the trained artificial intelligence (AI) algorithms to infer the damages as seen on the container image and classify them according to the damage classification. This process also helps generate a damage report along with the relevant evidence. This minimizes the need for the surveyor to meticulously document all the damages found, resulting in a significant efficiency boost at the container operational points.

That said, the hard reality is that the AI-based solution for container damage survey is less efficient (not able to capture damages consistently and completely) and lacks flexibility (any change in environment or physical conditions, etc., requires huge retraining). The root cause of this problem is a lack of sufficient training data, as well as of the digitalized ground truth for run-time validations. At the terminals, this is generally compensated by implementing stringent operational procedures (e.g., containers should move at a uniform speed and in a straight line while passing through the portal), stringent installation procedures (e.g., cameras need to be installed at a precise location and angle as even a slight change in camera orientation has a huge negative impact on the inferencing results), and investing additional effort in data reconciliation. The complexity of the problem further multiplies due to the lack of consistent, usable damage definitions across the industry, making ground truth definitions during training become less applicable in practice.

The SMART container damage survey

At ATAI, we have taken a drastically different approach, using computer vision and deep learning algorithms to develop the SMART container damage survey solution, overcoming the aforementioned challenges. This innovation has helped us address the ground-level, real-world challenges effectively. Seamless deployment of this solution,

even at the container exit points, helps to improve damage liability protection for the terminals. Key fundamental changes in our approach include the following.

The defining framework – driving uniformity in damage definition

Our solution provides a framework, based on the ISO 9897 standards, to define the damages according to the individual needs of every terminal. The framework allows a terminal to define the damage by specifying (a) the types of damages and localized classification names, (b) the significance or impact of the damage, (c) the rules of engagement for the given type and impact for a damaged container – this and many more.

Such an approach helps the terminals capture all the damages for a particular container and act on them as per their local definitions. This novel approach enables keeping the complete history of the container’s health, picking out the boxes that require special handling only when warranted. The SMART container survey framework also offers flexibility, enabling terminals to quickly adjust the damage definitions and operating procedures through configurational changes.

Raw images vs stitched images

Traditionally, the damage detection algorithms were fed with the stitched container side image as an input. Operational space constraints (full container view capture by camera frame requires a larger distance between the camera sensor and the subject or a very wide field of view sensor leading to image distortion) and the need to identify the location of the damage on a container (mapping and localizing) necessitated this approach.

However, stitching images to construct a full container picture is a computationally complex operation when we take the on-the-ground operational aspects into consideration (the container may be passing with varying velocities in front of the camera, the horizontal distance may vary across frames of a container, etc). This leads to distortions in the final stitched image if these dynamic variabilities are not compensated for. A distorted container image might lead to damage detection failures, false alarms, or damage misclassifications.

Contrary to this approach, ATAI has innovated in accurately mapping and localizing a given captured imagery on the overall container. This novel approach enabled the possibility of feeding the raw damage frames to the algorithms, resulting in more accurate detection and classification. It also allows for the usage of the damage images captured by other cameras from a different angle/perspective. While these images were ignored in the traditional approach (as they couldn’t be used for stitching), ATAI has put these images to work to create a more complete training data set, which also helps to provide damage evidence from multiple perspectives.

Accuracy vs miss rate

Unlike the common usage of AI in the terminal industry, where the focus is on improving the accuracy of the algorithms, the container damage algorithms need a different approach. This is because the damage-related data sets are not a balanced data set (the percentage of ‘damage’ vis-à-vis the percentage of ‘no damage’ is more skewed towards the latter category). Additionally, the cost of handling false positives (incorrectly inferring damage when none exists) is very high due to the manual nature of exception handling.

Because of the above, the algorithm training for damage detection should be more tuned towards increasing precision or sensitivity rather than accuracy. As such, ATAI’s SMART container damage survey system prioritizes minimizing the miss rate rather than solely focusing on improving accuracies.

Training data set – a realistic synthetic data creation framework

Access to the training data set for container damage poses significant challenges, as it requires capturing vast amounts of raw data, creating the ground truth, and ensuring the availability of sufficient data for each type of damage. This data creation process is time-consuming and sensitive to environmental and location variations, affecting the algorithm’s outcomes.

Recent advancements in technology have enabled the creation of synthetic data. However, the challenge with it lies in the accurate representation of damage and the loss of useful information in the process of transposition from a 3D to a 2D representation. Environmental and location factors further influence this loss of data. At ATAI, we focused on creating a 3D data set for damage data that incorporates environmental and location-specific variability. We developed a comprehensive framework for synthetic data creation using AI, image processing, and 3D rendering to generate the most authentic data set for container damage. By not relying on stitched images, our synthetic data avoids certain lossy transformations. Additionally, our data’s richness is enhanced by utilizing different perspective views of the same damage. The common damage model also enables us to deploy the SMART container damage survey at the exit points seamlessly, thus providing better liability protection for the terminals.

Damage validation mechanism – absence of the ground truth

In general, AI relies on probabilistic engineering and typically requires the digital ground truth to validate its outcome. As container damage detection algorithms cannot depend on the availability of the digital ground truth for ATAI’s container damage detection solution overcomes this model by (i) ensuring that algorithms are tuned for precision to ensure no damage is missed and (ii) through the availability of complete history from prior visits regardless of the terminal’s damage definition. This approach helps minimize the handling of false positives.

ATAI’s container damage detection solution overcomes this model by (i) ensuring that algorithms are tuned for precision to ensure no damage is missed and (ii) through the availability of complete history from prior visits regardless of the terminal’s damage definition. This approach helps minimize the handling of false positives.

Static model vs continuous learning

An essential and differentiating component of the container damage survey solution is the framework that enables it to incrementally learn from the accumulated feedback on the outcomes vis-à-vis ground truth.

This framework has also allowed us to create common models across various operational points of the container. For example, the damage detection algorithms used for a quay-side container survey are the same as the model used at the gate or rail sides of the container survey. Besides continuous learning, the framework has resulted in providing a consistent outcome across the terminal. It also helps the individual terminals to fine-tune the algorithms according to their specific needs and requirements.

Beyond ISO containers

The benefits of ATAI’s novel approach in addressing the challenges in a container damage survey at customer terminals have reiterated our leadership in innovation and customer-centric attitude. ATAI has extended this framework to encompass box damages for non-ISO containers, such as swap bodies, semi-trailers, and tank containers.

This novel framework allowed us to deploy and use common models across various types of terminals, including barge, intermodal, ro-ro, and inland container depots.

‍

Source: Baltic transport journal

https://baltictransportjournal.com/index.php?id=3336

‍

‍

‍