If you can't explain it, don't decide with it.
Customers, staff and regulators will ask why. POPIA section 71 even gives people a right to information about the logic behind automated decisions.
For the user
Why This Answer
Which sources it used, and what it was and wasn't able to consider.
For the person affected
Why This Outcome
The main factors behind a decision, in plain words, and how to challenge it.
For the auditor
How It Ran
Inputs, model version, instructions and logs, kept for each decision.
Source: popia.co.za ↗
Explainable, or not used.