A while back, we wrote a research paper on this very topic (I’ll drop the link below¹), and the question still stands:

Do we really need complex models all the time?

Over the past two decades, AI and ML have made their way into geotechnical engineering, and it’s been exciting to see.

Researchers have developed incredibly sophisticated models: neural networks, hybrid tree-based models, ANFIS, you name it. But here’s the real question: Are these models always necessary?

Many of these complex models come with some major downsides:

• Black-box nature: No clear equations, no interpretability.
• Computationally expensive: They take a relatively longer time to train.
• Prone to overfitting: Too complex for small datasets, leading to unreliable results.

On the other hand, simpler models, while not as flashy, offer huge advantages:

• Fully interpretable: You can actually understand and explain them.
• Fast to train: No need for high-end computing power.
• Less risk of overfitting: More reliable for small datasets.

What were the findings?

In our study, we used a publicly available dataset that had been used extensively in previous research. Many studies have applied complex ML models to it, reporting high accuracy but at the cost of interpretability and, in some cases, overfitting.

So instead of jumping straight into deep learning or hybrid models, we tried something different:
1- A simple linear regression model (70% accuracy, pretty solid!)
2- A 2nd-degree polynomial model, which outperformed all other models in accuracy.

The best part? Both models could be implemented in MS Excel, no need for advanced coding, heavy computations, or specialized ML frameworks.

What’s the Takeaway?

Before diving into complex models, we need to:

1- Assess the data first: Does it actually require a complex approach?
2- Start simple: If a straightforward model does the job, why overcomplicate?
3- Optimize before escalating: Feature engineering and better data preprocessing can often improve performance without adding complexity.

Now, this doesn’t mean complex models are useless, far from it.

But blindly chasing complexity doesn’t guarantee better results.

Sometimes, the simplest approach is the most effective.

What do you think? Have you seen similar cases?

1. https://ascelibrary.org/doi/abs/10.1061/9780784485347.039 ↩︎