Beyond the Framework

Introduction: Continuing the Dialogue

Following my previous post, “A Deep Dive into the PredicTri Framework,” In July, I had an insightful meeting with PredicTri’s founders, Yulia Yulish-Nechay and Ben Zickel. The discussion helped me gain a much deeper understanding of the philosophy behind their work and its implications for the industry.

A central theme quickly emerged: in an industry grappling with “black box” models, a technology partner’s culture of transparency is a critical product feature. Their candor about model limitations and commitment to open-sourcing their work is a direct answer to a systemic need for trust. This realization suggests they are building more than just a tool; it’s a new way of thinking about our profession.

The Model as a Diagnostic Tool for Business Reality

While PredicTri’s automated model competition is impressive for its accuracy, the discussion revealed a more profound application: it functions as a powerful diagnostic tool. The winning model doesn’t just yield a number; it provides a “fingerprint” of the portfolio’s behavior, offering deep insights into the underlying risk.

For instance, a “Non-Cumulative Residuals” model often wins for portfolios with high initial volatility that later stabilizes. In contrast, a “Cumulative Evolution” model is typically selected for less stable, long-tail lines. This transforms the actuary’s role from a modeler to a strategic advisor, who can now use these data-driven insights to ask sharper questions about underwriting and claims strategy.

XAI in Practice—From the Code to the Boardroom

The framework’s Explainable AI (XAI) provides a compelling solution to communicating complex results to non-technical stakeholders. Using Shapley values, it produces an explicit “receipt” for its calculations. For the Arius case study’s 2018 accident year, it showed precisely how a massive +12.9% adjustment led to the final ULR.

This transparent, additive explanation is reportedly “well-received and understood by non-technical stakeholders.” This de-risks the adoption of advanced models, transforming a “black box” number into an auditable calculation that satisfies regulators and builds trust with leadership.

Embracing the “Subjectivity Frontier” with Intellectual Honesty

The discussion of the “Buses uw” case, which involved messy, real-world data, highlighted the team’s transparent approach. When faced with data that violated a key model prerequisite, they acknowledged that the situation “currently requires user judgment” and confirmed they are already “working on alternative joint models” to automate the solution in the future.

This honest acknowledgment of the “subjectivity frontier”—where automation meets ambiguity—is a refreshing approach. It defines a symbiotic relationship where the tool automates what it can, flags exceptions, and relies on human expertise to resolve ambiguity, creating a virtuous cycle of improvement.

Redefining “Accuracy” with a More Sophisticated Lens

A model’s validation metrics reveal its core values. When asked why PredicTri uses the Energy Score over a more common metric like RMSE, the answer was a masterclass in clarity. While RMSE measures the accuracy of a single point estimate, the Energy Score evaluates the plausibility of the entire probability distribution.

This is not a minor detail. It’s a shift from asking “Did we get the average right?” to the far more important question: “Did we get the uncertainty right?” For an insurer, risk lies in the entire distribution. The Energy Score penalizes models that misrepresent tail risk, proving the framework’s design is coherent from its Bayesian foundation to its practical validation. This rigor is essential for high-stakes capital modeling.

The Path Forward

The PredicTri roadmap includes developing more flexible models and exploring advanced ensemble techniques. But their most compelling vision is their ethos. Their code is publicly available on GitHub (‘PredicTri’), and they are keen to “work with actuaries who are open to new ideas.”

Conclusion and Takeaways

The meeting with Yulia and Ben was incredibly insightful, reinforcing initial findings and adding a crucial strategic dimension. For any professional in this space, here are the key takeaways:

• The framework is a diagnostic tool: Its greatest strength is its ability to reveal the underlying nature of a portfolio’s risk profile.
• Its XAI bridges the gap to the boardroom: The transparent explanations make advanced models defensible to the highest levels of scrutiny.
• The team’s transparency builds trust: Their honest acknowledgment of the “subjectivity frontier” is a testament to their integrity.
• Their statistical rigor is impeccable: Their commitment to getting the uncertainty right is the cornerstone of sound risk management.

The logical next step for the industry is to move from the theoretical to the practical. Pursuing small-scale Proofs-of-Concept with real-world data represents a fantastic opportunity to continue this collaborative journey. It will be exciting to see the results.