Tuesday, September 06, 2016

,

A response to the media reporting on InterNations expat survey 2016

9/06/2016 Taiwan Explorer

When I saw the above headline in the Daily Mail, my BS radar started to blink. And it wasn't only the Daily Mail, it were also Focus Taiwan, Taipei Times, Forbes (obviously!), and several other news outlets that reposted the same thing (which includes Taiwanese, such as CNA, UDN and 天下雜誌). Then it all got viral on social media among some Taiwanese, but mostly among people in the English speaking Taiwan community. I'm generally suspicious of these viral lists that claim Taiwan was the best in something, so I decided to read the actual report. It seems that most news outlets have just re-summarized the summary of this report that was published on InterNations's blog. I have also signed up to InterNations to see what the site is about, so check that out first before you proceed reading.

• Link: "Who is InterNations, what do they offer to Taiwan expats?"

A summary of InterNations expat survey 2016

This is how Daily Mail summarized it:

A new study has revealed that Taiwan is the best country in the world for expats, while Kuwait is the worst. The results come from a report by the online expat community InterNations, which looks at everything from quality of life and personal finance, to job satisfaction and ease of settling indices. Taiwan was ranked number one out of 67 countries on the list, with Malta scored second place and Ecuador came in third, followed by Mexico and New Zealand.

Apparently this survey caused a minor controversy in Kuwait, a minister felt compelled to respond.



This is what InterNations says about Taiwan

The newcomer Taiwan has ousted two-time champion Ecuador to win this year’s survey. In addition to claiming 1st place out of 67 countries in the overall ranking, it is in the top ten for every individual index! Taiwan holds first place in the Quality of Life and Personal Finance Indices, impressing with the quality and affordability of its healthcare and the enviable financial situation of expats living there.

The Asian Tiger scores second place in the Working Abroad Index. Over one-third of expats in Taiwan (34%) are completely satisfied with their jobs, more than double the global average of 16%. Expats are similarly enthusiastic about their work-life balance (30%) and job security (34%).

This small island country also holds second place for overall satisfaction with life abroad, with 93% voicing their general contentment. Only Spain has higher ratings here. It comes as no surprise then, that a majority of expats in Taiwan (64%) plan to stay there longer than three years; with more than half of these (36% in total) even considering staying there forever.

Taiwan performs worst in the Ease of Settling In Index, although it still comes in a respectable tenth. Here, its first place in the Friendliness subcategory is evened out by much lower results in the Language subcategory, where it only comes in 45th. Nine in ten expats give the friendliness of local residents towards foreigners a positive rating, compared to only 65% worldwide. However, the language barrier does pose some problems. Only 23% overall agree that learning the local language is easy (global 37%) and about one-third (32%) are of the opinion that living in Taiwan without learning at least some of the local tongue is problematic. It seems that many expats anticipated this challenge, with 35% naming the language barrier as a possible disadvantage they thought about before the move.

Taiwan is the only country in the top three with enough expats living abroad with their dependent children for it to feature in the Family Life Index, coming in 8th out of 45 countries. In fact, 43% of the respondents there have dependent children living with them, over twice the global average of 21%. It does best in terms of the friendly attitude towards families with children, with 58% rating this as very good (39% worldwide). However, for both childcare and education, only 3% of expat parents completely agree that these are easy to afford.

This sounds all well and legit, but the biggest question for me is: How many expats in Taiwan have taken part in this survey, and who are they? I've read the whole report and InterNations doesn't disclose that information, but they do offer some general numbers, and the methodology they used. Based on that I will try to make some assumptions. This is a quote from the report (marked in red are my highlights, I want you to pay attention to this):

The Expat Insider 2016 survey was conducted by InterNations, with conceptual feedback provided by the independent market research company komma GmbH. The online survey ran from 18 February through 13 March 2016 and was promoted through the InterNations website, newsletter, and social media profiles — reaching out both to members of our network and expats in general. A total of 14,272 expatriates took part, representing 174 nationalities and living in 191 different countries or overseas territories.

The results of the survey are published here in the form of an overall country ranking of common expat destinations, and plenty of information grouped around five topical indices: Quality of Life, Ease of Settling In, Working Abroad, Family Life, and Personal Finance.

For a country to be featured in the indices and consequently in the overall ranking, a sample size of at least 50 survey participants per country was necessary. The only exception to this is the Family Life Index, where a sample size of more than 30 respondents raising children abroad was required. In 2016, 67 and 45 countries respectively met these requirements. However, in many countries the sample size exceeded 50 or even 100 participants by far.

For the rankings themselves, participants were asked to rate 43 different aspects of life abroad on a scale of one to seven. The rating process emphasized the respondents’ personal satisfaction with these aspects and considered both emotional topics such as the friendliness of the local population as well as more factual aspects (e.g. a ordability of healthcare) with equal weight. The respondents’ ratings of the individual factors were then bundled in various combinations for a total of 16 subcategories, and their mean values used to draw up the aforementioned ve topical indices.

These five indices, as well as the responses to the question: “how satisfied are you with life abroad in general?”, were then averaged in order to arrive at the overall country ranking. The Cost of Living Index, while allowing for a glimpse into expatriates’ perception of expenses in their country of residence, does not factor into the overall ranking.

Please be aware that the percentages in this report have been rounded to full integral numbers. In some cases, this may lead to all responses to one and the same question adding up to either 99% or 101%, instead of 100%. These instances are merely due to rounding and are not based on erroneous data.

Problem #1 with the survey: Anyone could take part

InterNations say: "Reaching out both to members of our network and expats in general" - this means a membership account on InterNations wasn't required. Not that it would prevent one from submitting false data, but I would say non-members are likelier to do so. Remember a year ago there was a survey done by CNN on Facebook asking people "Which destination has the world's best food?" and surprisingly Taiwan won - but only because Taiwanese voted for themselves. This is the reason I'm a bit suspicious of such polls and surveys where anyone can take part. I'm not saying this was the case here (I have no proof), but based on my findings, there are also a lot of Taiwanese active in the Taipei expat group (read my post). For example, in the largest group called "Taiwan Explorers Group" that has 247 members, 80 of them are Taiwanese (that is nearly 1/3).

Problem #2 I with the survey: Sample size is small

InterNations say: "For a country to be featured in the indices and consequently in the overall ranking, a sample size of at least 50 survey participants per country was necessary"). I find this to be the biggest problem of the survey. Let's crunch the numbers they provide and see what we get:

• 14,272 expats took part living in 191 different countries or overseas territories - That equals to an average of 74.7 people per country)

• 67 countries or overseas territories met the requirement of reaching the sample size of 50 participants - That means that 124 countries or overseas territories have not met these requirements, this equals to a minimum of 124 people (124 x 1) and maximum of 6076 people (124 x 49). I assume the truth is somewhere in between. Let us see the worst and best case scenarios:

Best case: If 124 expats were excluded from the survey, than that would make it 211 people on average in best case scenario (14,148 people divided into 67 countries).

Worst case: If 6076 expats were excluded from the survey, that would make it 122 people on average in the best case scenario (8196 divided into 67 countries).

The real number is most likely somewhere in between. I think if we take an average of these two values, we should be pretty close to the actual average of people per country, that were included in this survey: 167 people (I have rounded up the numbers). While being mathematically true, this is still an average, of course, and it won't really tell us how many expats currently living in Taiwan participated in this survey. However if we consider that only 20-30 people are on average publicly active on the InterNations Taipei expat group (based on my own research) and that currently Taipei's largest expat group has only 247 members (out of which 80 are Taiwanese), it's kind of reasonable to assume that we're somewhere in the range of 50 to 200 participants (also assuming that part of the participants were people who found the survey on social media). Personally I have a hard time believing that more than 150 Taiwan expats took part in this survey.

Problem #3 with the survey: We don't know much about participants

We don't know the age groups, the gender ratio, and nationalities of Taiwan expats who took part in this survey. We also don't know what work they are doing, what are their annual earnings, and how many years they are living in Taiwan. We also don't know how many are members of InterNations website, as I mentioned before. If we knew this, it would give us a much better idea about who this group of expats in the survey actually represent. InterNations, I assume, has this data (at least some of it), but they are not sharing it publicly. Why not? Maybe because "Taiwan named best expat destination in the world" wouldn't sound that credible? What is the motive behind InterNations not disclosing this information? I can only speculate at this point.

Problem #4 with the survey: Taipei is not Taiwan

InterNations as well as the media reporting on this, are conflating Taipei with Taiwan, which is indicated in this quote: "About nine out of ten expats in Taiwan (91%) are overall satisfied with the local transport infrastructure, compared to 63% globally." We all know Taipei has an excellent public transport infrastructure, but when you go anywhere else you often have to rely on a scooter and car, if you own one. Buses and cabs are less, and a subway system is only present in Kaohsiung (however it's much much smaller compared to Taipei). There's currently only a Taipei expat sub-group found in InterNations, however Taipei doesn't equal Taiwan.

In conclusion


Screen grab from the South China Morning Post.

Unfortunately media these days doesn't always do their job, what matters is attractive headlines for the clicks. Time to do journalistic research seems to be very constraint, what matters are quick stories, fact checking is not always a priority. InterNations does these annual surveys to achieve exactly what happened here: Free publicity. A lot of people who have never been to Taiwan will now erroneously think that this is an expat paradise, something that could not be further from the truth. InterNations business model consists of paid memberships + offering a platform to advertisers. In order to grow their business, they have to grow the total number of members and the total number of their social media followers. Nothing wrong with that, but one has to take findings of such survey with a pinch of salt. Only a small % of the 2.3 million members InterNations claims to have are actually active on the site. And only a tiny % of expats in Taiwan are represented in this survey. As of July 2016, Taiwan had in total over 600,000 expats from Indonesia, Thailand, Vietnam and Philippines (source). These expats are typically blue collar workers, found in various factories and homes as domestic helpers and caretakers, and represent the largest group of expats in Taiwan by far (typically around 90% of all expats). If we assume that the total number of expats in Taiwan right now is around 650,000 (in 2015 it were close to 638,000 according to ROC Yearbook), then 150 expats represent merely 0.02% of all expats, which is an insignificant sample. Even if the number of Taiwan expats participating in the InterNations 2016 survey were 5 times larger, it still would not change this fact.

Stay tuned for the 3rd part of this series, where I will write about the challenges expats face in Taiwan right now.