Statistagasm: The Statistics of Fate

Let big data read the palms of individuals....

Can you predict your own future? The question sounds horribly sawed and straight out of a poorly written prophetic guide. But, let’s rephrase it in a more specific way, “Where would you think you would be in a five years’ time?” And most important, what should follow is “Can you predict it, or know something about it in reality?” There are a few common responses to this question, which say something about how people, in general, view fate or the course of events in their lives. Some of them which I get frequently, depending on the type of person, are “No, future is not predictable”, “Yes, if you believe in your dreams, you can be there" or if you are religious “Yes, only God knows it”. A more interesting segment, however awaits when you ask them “Can you predict how many people would give the same answer as yours, beside the other ones?”

Well, as it unfolds, this might not be a very difficult task. In that, we can reasonably estimate how different people are going to respond to the first part. The response obviously relies on many facets such as the amount of control one might feel one has at a given time. But more important, it falls onto larger scaled phenomenon, such as culture and socioeconomics. A person sitting in West, where the cultural focus is on individual effort and ‘chasing your dreams’, is likely to think that future is predictable while someone, who happens to be born in a country plagued by poor governance would rather believe that outcomes in life are unpredictable.  It is not a random occurring that someone has chosen to respond one way or the other to the questions above, but instead it is embedded in a meta-statistical structure. Of course there are outliers, and of course there is a margin of error, which is precisely what differentiates statistics from prophecies, which can vary from a lot to negligible depending upon what it is which we trying to predict. If you are digging in to predict the response of a single individual, given that you know what is there to know about that person, then the margin of error would be very high. But if you are engaging to know the response of a particular group, or population, the predictability would sharply jump. In this matter, the lives of humans closely resemble the natural world, which, as you go at the smaller levels, become more and more unpredictable, and as you slide towards larger levels, let’s say at the Newtonian level, you begin to decipher patterns and laws.

So where will you be in a five years’ time? Or, how much will you be making at your job? Which life partner would you have? And how much happy would you think you would be with them? These are no questions for palm readers or crystal gazer, their answers clearly lay with good statistics and good philosophy. For example, dating patterns show that the US millennials are getting into relationships earlier than their ancestors were and are likely to go through an average of two long-term relationships and several mismatched dates and heartbreaks before arriving at “the one”. So, it should not come as a surprise that some would feel their relationship history is un-normal, when that is what most people are playing  around like. More interestingly however, on online dating platforms, people are more likely to hit those who belong to the same race, despite answering “it does not matter” in their response boxes, and to opposite sex, despite describing "bi-sexual" as their sexual orientation. It is easier and more useful to analyze online dating data than to take surveys and responses from people in the field, because the online dating show clicking and email patterns which reveal what is really going on, instead of merely asking a sample which often results in inaccurate responses. It seems like, that the fate happens to be rigged while arranging partners because it somehow always manages to set dates with those who fall near your class bracket, and also somehow with those who share your skin color.   

The performance in education and the amount one makes at his job are other facets of the larger scheme of things, which too seem to be rigged. This has been repeatedly shown, when we see that middle-classers outperform lower-classers in schools consistently no matter which country or culture. In South-Asian societies, the cultural focus in academic achievement is primarily ‘the more you hard-work, the more pleasant your grades will be’, something which is both disastrous and wrong from the view of policy-making. Differences in income of the households has profound impact on educational performance of the children in those households, translated from cumulative factors which seem minor to ordinary people. For example, research shows that lightening and temperature at the place of study influences concentration and therefore learning. Proper lightening and study place is recommended during focused study hours. Apart from that, getting someone to monitor your performance significantly reduces procrastination and improves lacking. Middle-class children are likely to have educated parents who keep track of their children’s performance, visit schools often, meet teachers, and are likely to hire coaches if their children underperform. They are also, likely to closely guard their children’s television time, computer games time, and are likely to encourage them to study beyond the scope of the syllabus. It seems like the “hard-work” phenomenon does not work quite well, when you have a noisy household, where you are interrupted every few minutes to do the house chores and where you don’t have any authoritative agency to keep you motivated. There is no co-incidence that the sprawling majority of the top colleges and universities are the middle and upper-classers. The drumming whole of the values of hard-work which are constantly bombarded into the popular ears, would work fine if everyone played by the same rules.

There were just some of the major superficial factors which contribute to educational performance, scratching the surface would uncover more significant variables, for example psychological and cultural ones. In households, where partners are in emotionally stable relationships, children are likely to perform better in the educational pursuits than in households with less stable partnerships. Children raised by single mothers who most often belong to lower rungs of society or adoption homes, are not only going to significantly underperform in schools, but have a high dropout rate and are also very likely to engage in criminal activities, such as drugs, violence and abuse. This has important implications for policies concerning abortion and birth control. The Donohue–Levitt hypothesis, shows the impact of legalized abortion on reduction in crime rates. People who repeat the ‘let the fate decide’ mantra should very well look at the probability of someone from an economically meek and psychologically unhealthy surroundings ‘making it’ around. What they don’t realize is that the fate which they so earnestly assert, is basically the structural forces which decide the life outcomes of individuals to a very large degree. This continues itself into the work lives, and an even greater gap in the incomes of individuals.

But these were the ultra bolded lines on a map, which many would feel are obvious to highlight (even though that’s not the case). What about the thin lines or subdivisions? In other words, how come inter-class, inter-group factors lead to differences in the incomes earned? Well, as it might unweave, these too are not that random. HBR often publishes studies showing salary differences between managers in the same occupation, in the same industry and even with same qualifications. There is of course, a whole complex web of factors feeding into one and another, but they are not non-traceable.  On individual level, for example, person-job fit can be an important factor in one’s job success which is the fit of personality traits of an individual with the requirements of the job. Now, whose fault is this that your genes happened to express themselves in a certain way to make your personality fit or unfit for the job you happened to find. You cannot possibly trace the genetic and phenotypical characteristics of individuals to predict whether they have the ability to smile most of the time, if they chose to be frontline managers. It might seem absurd, as it bears witness to a previous analogy, that on smaller scales such as at individual level, things become very unpredictable. There is still, however, a probability which determines the chance that your genes happened to be arranged in a certain way, to make up your personality traits both naturally and by training, which happened to be suitable for your job, and which now earns you way above mean of your industry, occupation and geography, but it would be ridiculously difficult to calculate.

Gleaning at the big picture, the crime and health statistics, in a similar way, help us look at the wider inferences. For example, measuring and comparing weight patterns across the world show that obesity has a high incidence rate in developed and developing countries. But this is not the end of story. In rich nations, people who are poor financially and socially, are more likely to be obese. Not only that, but weight is also found to be unevenly distributed among sexes, with women typically being more obese than men. There is also a correlation between higher gender inequalities with higher obesity of females. Now, all of this does not quite fit into the over-consumption of food theory because there is a stark difference between the quality of food consumed by different brackets of society. Lurking beneath are the high levels of stress, which cause hormonal disturbance which affect fat storage and metabolism. The incidence of diseases or crimes for that matter, are not population variant but also, more important, historic and structural. The incidence of violence and terrorism, as is now popular knowledge, a characteristic of poor and ill developed societies, but it is too unevenly distributed among areas and groups. The likelihood of violence is a lot more, within the same country, in areas historically disputed. Those areas are also likely to be least well off than the rest of country. Now, the cause and effect can work both ways, whether poverty leads to violence or violence perpetuate poverty, but the overall message is that no problem is isolated, one problem intricately feeds into the other creating a vicious cycle of all-time misery.

 

Now, let’s dissect the portion which this post is originally about. The statistics of fate. The probability hits which, many would say, you cannot control such as being born in a particular nationhood, race, or bracket, which is supposed to ‘just happen’. Well, by control, one should mean that an individual cannot control these occurrences, because when one wear telescopic glasses and affords a bird eye of view of the statistical terrain, the ‘controllable’ deem obvious. The probability of you being born in a country which are ranked high on development index, is basically the children born in those countries every year over the children born all around the world every year. And since, high ranked countries have low reproduction rates, the probability would be quite low. This might be an inaccurate measure because children being born are not equal to children who survive, and in countries who rank low on development index, the mortality rates are high but the population growth rates too are high despite more children dying, so there is more likelihood of being born in medium and low ranked countries. 

As you might have already guessed, this probability depends on the sprawling global inequality. In a world with more equal distribution of wealth, the probability of a child being born in downcast surroundings would be droopingly low. Yes, it is not the child’s fault in where he or she happens to be born, but it is very much the fault of the larger world dynamics which clearly determines the happens-to’s.  Of course, it does not end there, there are numerous systematic gaps which pronounce the decision of one fate over the other. Subsequently, it trickles down to thinner and less visible lines of the map, as the class, race, status, gender and ethnic gaps are bridged on. However, many would throw the query that what about the incidences which are not apparently the result of structural gaps, such as earthquakes or tsunamis, which are totally unpredictable and thus out of control. Well, of course that is true, but that is not how we are looking at the statistics. The problem does not lie in how many disasters hit the population, but actually how they impact it. The disasters are likely to do more damage in resource stricken areas because their rehabilitation take much longer, and sometimes is never initiated. The trick here, is not to never expect bad things, but to have a strong coping mechanism against those occurrences.

The underlying power of statistics is truly unthinkable, and touch the secluded grooves of information where ordinary eye cannot intrude. Maybe one day, we get to the position of predicting the unpredictable, as both the sources and quality of data improves. A time when big data is used to read and predict the fates of masses, a juncture where science does replace prophecy.  The whole anagram of ‘fate’, as some veiled plan, has consistently been used as excuses against bad governance, bad policies, bad economics, bad socials and most important, bad information and bad philosophy.

Between Hell and Heritage

The ingenious way how cultures trademark their intellectual heritage

Photo by Sam@Carlton School

Around the twelfth century, amidst the brooding courts-men of the Spanish Caliphate, a meekly figured man with a kind regard for both Aristotelian philosophy and popular ideals of his time, spoke on the vulnerability of the nature of truth. This, however did not earn him much respect, instead he was disdainfully thrown out of the court, and his philosophy labelled as impure and corrupt. That man, which is today hailed as Averroes did struggle quiet, in his time, to avoid problems with the Islamic authorities, moreover this was his unlucky moment.

A sophisticated view of the past would reveal that there is a plethora of similar instances, when the intellectual minority of a community or civilization got into screeching friction with the mass populace. This is not confined to only philosophy or the study of the natural world, but also art, poetry and music. Some of it, do make itself into the popular consciousness, but some of the more non-digestible forms remain seated on the shelves, yellowing and rotting with time, and only occasionally discussed in the academic squares.

Averroes, or more precisely, Ibn-Rushd was not alone in the staggering ‘clash of the philosophies’, there were many who were received and seen with contempt by the larger population in their own times, and even in times today. Amongst them, the more famous or rather infamous were Ibn Sina and Al-Farabi, who were considered “too deviant” that some scholars discussed their status as Muslims. Ibn al-Qayyim (1292- 1350) and other Islamic scholars, clearly declared them as extreme and disbelieving.

The history of intellectual developments, is not simply the history of the people. It is the history of class, caste, economics, and politics. To say that Egyptians made the pyramids, Indians wrote the Kamasutra, Westerners drafted democracy and freedom of thought, and Muslims made algebra would be an exceedingly immature and unsavory assessment of the past. Today, many Christians in the West would proudly boast about the “civilized” modes of thinking possessed by the westerners but they would surely not like to talk about how historically those civilized modes of thinking, which they take for so much granted came about in the first place. And they assuredly would not like to mention the incalculable exiles, torments and blood sheds of the free-thinkers and rationalist who gave and perpetuated the modern ideals by putting the Church’s teachings under scrutiny. But the way cultural trademarks rulebook works, the “credit goes to the whole Western civilization! Big round of applause, everyone”

The Kamasutra and other pieces of writings from the Gupta and subsequent periods, popularly regarded as the golden age of Hindu civilization too divulges a similar picture. The philosophical and sensual writings were the product by and for the elitist rulers and courtesans. The clergy and religious perception of the wider population resembled nowhere close to it. Indians did not write the Kamasutra, the people who had leisure, time in vain and tax money in their pockets did. The masses simply ploughed land, sowed seeds, prayed to gods and harvested food, only to give it away in the taxes.

                                                                                                                                        We don’t know how many Averroes were expelled or scorned at, and how many got into serious troubles, but it is vehemently celebrated as the “golden age of Islamic civilization”. Only a few however, will ever bother to dig and know, for example what al-Razi, whose contributions in chemistry and medicine were unmatchable at the time, thought about miracles and revelation.  And what his “heretical” works such as The Prophet’s fraudulent tricks, or On the refutation of revealed religion undertake about the wider orthodox conceptions.

This takes us back to the ingenious way how cultures mark their intellectual territories through panoramic generalizations and uniformity, throwing the scrupulous contextualization of history, politics, economics and socials far off in the reality dustbin, never to be hunted and picked out again.