Deep Science: AI Honesty in Medicine, Climate & Vision

The Rapid Pace of Machine Learning Research

The sheer volume of research papers published today makes it impossible for any single person to stay fully informed. This is particularly true within the rapidly evolving field of machine learning, which now impacts and generates research across nearly all industries. This column serves to highlight noteworthy recent discoveries and papers – with a focus on, but not limited to, artificial intelligence – and explain their significance.

Bias, Cheating, and Data Integrity in Machine Learning

This week’s selection centers on identifying and addressing biases, detecting deceptive behaviors, and uncovering flaws within the data used to train machine learning systems. However, we begin with a visually striking project originating from the University of Washington and presented at the Conference on Computer Vision and Pattern Recognition.

Researchers developed a system capable of recognizing and predicting the movement of fluids – water, clouds, smoke, and others – within photographs, effectively animating them from a single static image. The resulting visualizations are remarkably compelling.

Image Credits: Hołyński et al./CVPR

The rationale behind this work is multifaceted. The future of photography is increasingly code-driven, and enhanced camera understanding of the visual world will enable more sophisticated image capture and manipulation. While realistically simulating river flow may not be an immediate priority, accurately predicting motion and common visual elements is crucial.

Ensuring Machine Learning Systems Perform as Intended

A fundamental question in machine learning is verifying that a system is actually accomplishing its intended purpose. The history of “AI” is replete with instances of models that appear to perform a task without genuinely doing so – akin to a child superficially tidying a room by concealing items under the bed.

This issue is particularly critical in the medical domain, where a malfunctioning system could have severe consequences. A study conducted by UW researchers revealed a tendency for models in existing literature to engage in “shortcut learning.” These shortcuts can range from simple correlations – such as basing an X-ray risk assessment on patient demographics instead of image data – to more complex dependencies on the specific hospital environment where the training data was collected, hindering generalization to other settings.

The team discovered that many models faltered when applied to datasets differing from their training data. They advocate for increased transparency in machine learning – opening the “black box” – to better identify instances where systems are circumventing proper procedures.

Image Credits: Siegfried Modola (opens in a new window) / Getty Images

Validating Climate Models with Machine Learning

Conversely, climate modeling presents a challenge of immense complexity, requiring supercomputers to simulate even small-scale atmospheric and aquatic movements over extended periods. Machine learning systems can simplify this process by predicting future states based on historical data. However, a key question arises: is the system truly modeling climate dynamics, or merely making statistically probable predictions?

A study initiated at the University of Reading yielded encouraging results, demonstrating that these systems genuinely model the underlying climate factors. Co-author Valerio Lucarini stated, “In some sense, it means the data-driven method is intelligent. It is not an emulator of data. It is a model that captures the dynamical processes. It is able to reconstruct what lies behind the data.”

Explainable AI for Flood Prediction

This level of confidence is invaluable for applications like flood prediction. A project from Lancaster University, building on earlier versions that lacked similar assurance, aims to create a faster, more accurate, and – crucially – explainable flooding model. We can anticipate that this emphasis on “here’s how we know what we know” will become increasingly prevalent in AI systems with the potential to cause harm.

Detecting Bias in Student Dropout Prediction

Predicting student dropout rates presents a different kind of challenge. Algorithms can inadvertently rely on spurious correlations. Researchers at Cornell investigated whether including protected demographic information – race, gender, and income – would affect these models. Fortunately, they found no systematic bias. In fact, the team recommended including this data to create a more comprehensive and inclusive assessment.

Modeling the Human Brain with Neural Networks

Simulating neural networks – those within our brains – using artificial neural networks – those within our computers – appears intuitive, but is surprisingly complex. While the latter are inspired by the former, they are not inherently well-suited for accurate simulation.

Image Credits: EPFL

However, brain neuron activity can be monitored and predicted like any other complex system. Researchers at EPFL are pursuing a project to develop the foundations for visual prosthetics by modeling the visual cortex of blind individuals. Accurate prediction could reduce the need for frequent and invasive testing, allowing for simulation of adaptation based on early indicators.

Smart Homes for Dementia Care

Individuals aging with conditions like dementia require significant care, often exceeding available resources. A recent study by UC Berkeley researchers suggests that smart home devices and machine learning could provide assistance.

Image Credits: Robert Levenson / UC Berkeley

Homes were equipped with sensors to monitor activities like faucet usage, bed occupancy, and door status, establishing a baseline of normal behavior. Deviations from this baseline could signal confusion or distress, alerting caregivers. This approach reduces caregiver burden and provides a flexible, responsive technological layer.

Improving Fairness in Image Datasets

Google recently revisited a large image dataset to assess fairness metrics. The dataset comprises 9 million images, including 100,000 with people, and the goal was to evaluate the consistency and impartiality of labels and bounding boxes.

Examples of new boxes in MIAP. In each subfigure the magenta boxes are from the original Open Images dataset, while the yellow boxes are additional boxes added by the MIAP Dataset. Image Credits: left: Boston Public Library; middle: jen robinson; right: Garin Fons; all used with permission under the CC BY 2.0 license.

A second review identified tens of thousands of previously unlabelled individuals and refined the representation of age and gender. Instead of labeling “boy” or “woman,” labelers now identify “person” and then add gender and age presentation labels. This inclusive approach is more practical, as systems are more likely to search for “people” rather than individuals with specific gender presentations.

As the researchers concluded, the revised dataset is more inclusive and improves the overall quality of the data, reducing the risk of perpetuating human biases in machine learning systems.