Financial models such as discounted cash-flow, Value at Risk. Black Scholes, corporate ratings models and copulas continue to be used to guide investors despite large forecast errors. Ekaterina Svetlova’s foundation is that the forecasts from financial models as a class will never be precise because economic uncertainty is large and unknown (radical).  In her own words, “that ideal is too far out of reach.”

I agree with this, but if you subscribe to the view that financial models would forecast precisely if only they were done properly, you probably don’t want to read this book or even the rest of this review.

Because this point is so critical to the book, I would have liked to have been lifted out of the singular type of evidence presented and be shown estimates of the scale of uncertainty in outcomes. One case in the book is that of Amazon where early evaluators had to decide whether to categorise the company as a bookseller or something else, with radical uncertainty in this categorisation. I would like to see the scale of the challenge faced by financial models ranked against errors from other model-based forecasts in modern society such as weather forecasting, predictive analytics of consumer behaviour, geologists’ forecasts of lucrative oil or mineral deposits, futures markets, or legal decisions, election decisions and so on.

But if we accept that uncertainty faced is huge and, beyond some point, untameable, why are models so popular? Here we get to the novelty of the book. The book is not a textbook about models themselves but how they are used.

Models are blamed for the failures. But are they the real villains or scapegoats for users? Why do financial forecasters get paid anyway? As I read the book, the six lines of Philip Larkin poem’s As Bad as a Mile ran through my mind. The poet throws an apple core into a dustbin and misses, and as he sees the core skidding across the floor, he imagines failure spreading up the arm into himself.

Svetlova gets her best evidence from speaking to practitioners who use financial models, and usefully this includes whole teams or communities. The first dramatic clue is that practitioners are completely aware of the failings of their models.

As she parts the foliage of model marketing, we see the models’ true purposes in a financial eco-system that is deluged with data but also more complexity. She elegantly shows how models spur “active decision making.” When a group runs a model, they must make quantitative decisions about inputs, evaluate outputs and explain forecasts. The group members are thus freed from the “paralyzing effects of having to act in unpredictable environments” (Jens Beckert, Imagined futures: fictional expectations in the economy) and therefore, the model serves as an “ice-breaker”, like a game of Twister at the start of a dinner party.

The implications of this secret life are startling and many and I will leave the reader to read Svetlova’s conclusions and draw their own.

Not all is good. Svetlova probes into the paradox that practitioners often present models’ forecasts externally as precise when they clearly are not. My own explanation is that practitioners have this habit from university life where only statistically significant results are rewarded. The narratives we hear in applied economics classrooms have only two endings: a p-value of 0.05 of less or abject failure. (See this excellent Moving to a World Beyond “p<0.05” by the American Statistician for solutions to this problem, including encouraging academics to think like a former generation of practitioners). For whatever reason, the huge uncertainty surrounding our calculations seems to be a shameful secret that we must cloak with over-confidence and opinion. All the wonderful things we can do to narrow down the range and derive triggers for different paradigms is less valued.
Meryn King’s End of Alchemy (Chapter 4) is also frank about radical uncertainty and is a companion to this book. King splits coping solutions into categorising problems, rules of thumb and a narrative and Svetlova writes about how models help with these. See also William Allen’s recent SPE review of “Uncertain Futures” by Jens Beckert and Richard Bronk.

Svetlova has written a thought-provoking book with an important message.  My only complaint is that while the interviews are revealing and there are diagrams relating the many concepts, I think the book still betrays too much its likely origin as a research report or dissertation. It is a shame that more could not be done to make this book accessible and entertaining.  

How will I use it? I will do more to tell my clients, colleagues, peers and especially graduate economists about the secret life of models and make them think about how we might do things differently. Please wish me luck.