The Language-Based Recruitment of Migrants to Online Surveys with Facebook Advertisements: A Comparative Assessment from Three Geographical Contexts

Introduction

Response rates across different modes of survey data collection have been in decline for more than two decades (Curtin, Presser and Singer 2005; de Leeuw and de Heer 2002; Sheehan 2006; Stoop 2005). The modern pace of life, increasing urbanisation and the public’s general saturation with and scepticism towards surveys are among the reasons why traditional strategies for participant recruitment, including posted questionnaires, telephone interviews (Curtin et al. 2005) and email surveys (Sheehan 2006), have become less effective. At the same time, rapidly expanding Internet connectivity, reaching nearly universal coverage in the most developed countries and growing fast across the rest of the world (Internet World Stats 2021), has allowed online social networks to grow rapidly in the past decade. This development encouraged researchers to recruit informants online (Kapp, Peters and Oliver 2013; Rife, Cate, Kosinski and Stillwell 2016).

The trend towards the recruitment of study participants via online social networks has proved successful in health, medical and psychosocial research (Kayrouz, Dear, Karin and Titov 2016; Pedersen and Kurz 2016; Thornton, Batterham, Fassnacht, Kay-Lambkin, Calear and Hunt 2016; Whitaker, Stevelink and Fear 2017). In particular, recruitment through online social networks has benefited research on hard-to-reach social groups for whom anonymity is crucial, such as sex workers, victims of family violence, HIV positive persons (Yuan, Bare, Johnson and Saberi 2014), young marijuana smokers (Ramo and Prochaska 2012), or supporters of conspiracy theories (Iannelli, Giglietto, Rossi and Zurovac 2018). Despite its potential for participant recruitment among hard-to-reach populations, the use of online social networks for recruiting participants to migration research is relatively new. This may seem surprising given that ethnic minorities and immigrants in Western countries have traditionally been underrepresented in social and demographic surveys for a variety of reasons, among which inaccurate or incomplete sampling frames, language barriers, irregular residency and cultural bias towards participation (Eisner and Ribeaud 2007; Feskens 2009; Feskens, Hox, Lensvelt-Mulders and Schmeets 2006). Another explanation of why online research overlooked migrants in the early 2000s might be the conception of them as less likely to have regular access to the Internet or sufficient computer literacy skills to participate in online surveys (Feskens, Kappelhof, Dagevos and Stoop 2010, cited in Kappelhof 2014). This, however, has changed dramatically in the past decade due to the mobile Internet’s rapid penetration even across developing countries (Internet World Stats 2021). Making Skype calls, sending mobile remittances or simply staying in touch with family and friends through Facebook, Snapchat or Instagram has reportedly become an important part of migrants’ daily lives (Cassar, Gauci and Bacchi 2016; Hunter 2015; King-O’Riain 2015).

Facebook’s social network as a platform for migration research

In the past few years, scientists have begun to explore the potential of Facebook for recruiting migrants to online research. Founded in 2004, Facebook is the world’s largest online social network, with about 2.85 billion monthly active users (MAUs) and 1.88 billion daily active users (DAUs) as of March 2021. The company is also the world’s largest advertisement platform and, in the first quarter of 2021, reported USD 25.44 billion in revenues from advertising (Facebook 2021). Facebook features a targeted advertisement platform – Facebook Ads Manager – which provides advertisers and researchers with tools to customise target audiences for their adverts. Researchers can use a wide range of criteria – for instance, users’ socio-demographic characteristics (e.g. age, sex, education, language, country of residence and many more) – and special interests to tailor their ads for specific audiences. Based on the target criteria, Facebook Ads Manager then estimates Potential Reach – in other words, the size of the target audience that the ad could potentially reach. The estimation of Potential Reach considers many factors, such as ad targeting criteria and placement locations, how many people have been shown ads on Facebook Products in the past 30 days and what type of content people interact with on Facebook (such as liking a Page), etc. Facebook warns that Potential Reach is not an estimate of how many people will see the advert, as this may change with time; nor is it designed to match census population or other sources. Notwithstanding, this rich and virtually free source of data has proved useful in research on stocks and flows of migrants (Dubois, Zagheni, Garimella and Weber 2018; Palotti, Adler, Morales-Guzman, Villaveces, Sekara, Garcia Herranz, Al Asad and Weber 2020; Zagheni, Weber and Gummadi 2017).

However, given that Facebook Ads Manager does not provide advertisers with a straightforward marker of users’ migration background, migration scholars must resort to strategies that aim to infer such information with the help of other proxies, such as user’s languages, interests and previous residence abroad, known as ‘Lived in – country’ criterion. The latter seems to have been prioritised in recent studies. Through applying the ‘Lived in – country’ criterion, formerly known as ‘Expats – country’,¹ Pötzschke and Braun (2017) succeeded in recruiting a sample of 1,103 Polish migrants from Austria, Ireland, Switzerland and the UK without using incentives and with a relatively low budget. Their study proved that the geographic targeting of Facebook advertising based on users’ IP address is a reliable tool for sampling Facebook users in specific countries, claiming that 96 per cent of the recruited sample matched the target criteria. Moreover, they reported that almost half of the participants in their Facebook sample would not have been reached had they chosen traditional sampling methods. In another study, Dubois et al. (2018) combined user’s language (Arabic) with the former catch-all category ‘Expats(all)’ as a proxy for migrants in Germany from Arab League countries. Similarly, Palotti et al. (2020) used the criterion ‘Lived in – Venezuela’ for estimating the recent stocks and flows of Venezuelan migrants in Latin American countries in the aftermath of the Venezuelan humanitarian crisis. In a more-recent study on political participation among Argentinian, Polish and Ukrainian migrants in Germany, Spain and the UK, Ersanilli and van der Gaag (2020) experimented with different strategies, using both language, interests and ‘Lived in – country’ criteria.

Despite some progress in this direction, issues related to accurate and reliable targeting persist in the field. For example, some studies seem to take the estimated size of their adverts’ Potential Reach for granted. Such was the case for Pötzschke and Braun (2017), who used the ‘Expats – Poland’ criterion for the recruitment of Polish migrants in four European countries and reported Potential Reach figures ranging between just 27.8 per cent and 47.2 per cent of the official Polish citizen population in their respective countries of study. Similarly, Ersanilli and van der Gaag (2020), who targeted Polish migrants in five European countries using a similar strategy, reported significantly different Potential Reach estimates, depending on whether or not an additional criterion, known as ‘Away from hometown’, was applied. Hence, after they had removed this latter criterion, their figures jumped from 110,000 to 320,000 users in the UK and from 70,000 to 290,000 users in Germany (E. Ersanilli, personal communication, 21 May 2021). Other studies do not report Potential Reach altogether. Dubois et al. (2018), for example, compared the distribution of refugees (per km²) in the 16 German states from a report by the Brookings Institute with the distribution of their advert’s targeted users per state. Despite reporting a near-perfect correlation between the two distributions (Pearson’s r = 0.99), the authors do not provide the Potential Reach either for Germany as a whole or for its states. In addition, there are other weaknesses in their approach. Firstly, the data from the 2016 Brookings Institute Report, which the authors refer to, concern all refugees accepted in Germany in 2015, whereas their selection criteria, set in 2018, targeted only the Arabic-speaking ‘expats’. Secondly, since Facebook does not differentiate between the categories of migrants and expats, the Potential Reach estimates, based on the ‘Arabic + Expats(all)’ specification, included both refugees and other categories of migrants. In other words, comparing the Potential Reach estimates of Arabic-speaking ‘expats’ with the statistics on all refugees in different reference periods may have led to inaccuracies, making assessment of their targeting strategy challenging. Similarly, Palotti et al. (2020) found a near-perfect correlation between the geographical distribution of their Potential Reach estimates of Venezuelan migrants and the data from the January 2019 report from the Regional Inter-Agency Coordination Platform for Refugees and Migrants from Venezuela (R4V). However, they themselves pointed out the vulnerability of their strategy, noting that their Potential Reach figures of Venezuelans living across Latin America plummeted suddenly from 3.2 million on 14 March 2019 to just 2.4 million on 17 March 2019, after Facebook’s algorithm ‘Lived in – country’ had been updated around 15 March 2019. Such issues raise legitimate concerns over the effects of applying different target criteria to the coverage of the targeted migrant population and, related to it, potential bias in the composition of Facebook-recruited samples. It requires researchers to better document and reflect more critically on the accuracy of their targeting strategies using different proxies for users’ migration background.

Goal and structure of this study

This article is part of a larger migration research project which aimed to recruit survey participants from among Polish migrants who have settled in Norway, Sweden or the UK, specifically after the 2004 EU enlargement. When the original recruitment strategy, described later in the text, failed, a sponsored advertisement with Facebook Ads was launched. Instead of applying the ‘Lived in – country’ criterion, commonly used in similar studies as a proxy for migration background, this study chose users’ language as a marker of their migration background. The study succeeded in collecting 5,639 complete and incomplete responses during the advertisement period of 50 days, where 93 per cent of the responses originated via Facebook. The main goal of this article is to assess how well this language-based targeting covered potential respondents and how the socio-demographic composition of the recruited samples reflected the demographic profile of the targeted migrant population across the three immigration contexts. The article is organised as follows. The next section describes the design and logic of the two recruitment strategies used in this study. Then the results of the Facebook recruitment strategy are presented in terms of its Potential Reach as well as the demographic composition of the recruited samples. In the discussion, data from official sources of statistics are used to critically reflect on both the target estimates of the Facebook ad and the socio-demographic composition of the Facebook-recruited samples in the three countries of study. Zooming in on the UK context, the targeting results of this study are then compared with those found in Pötzschke and Braun (2017) and Ersanilli and van der Gaag (2020) who used ‘Lived in – country’ criterion to target Polish migrants in the UK. Finally, the last section summarises the main findings.

Methods

Recruitment strategies

In this study, two strategies for participant recruitment were used. Given the project’s goal to recruit Polish migrants living in Norway, Sweden and the UK, the initial strategy aimed to advertise the study in transient places, such as airports or onboard international flights. Thus, in April 2017, when Polish migrants travelled to Poland for the Easter holidays, a 3-week advertising campaign was launched with a low-cost airline which operated 61 distinctive routes and had over 250 weekly departures to Poland from Norway, Sweden and the UK. The campaign targeted all passengers flying on these routes between 1 and 21 April 2017 and had a total cost of NOK 176,000 (approximately, US$ 21,000 based on the payment date’s exchange rate). The ad consisted of a picture (see Figure 1) automatically placed in 113,114 self-printed boarding passes whenever the passengers checked in online and, additionally, in 113,082 booking confirmation emails whenever a relevant booking was made during the advertising campaign. The participants were promised free participation in a draw for prizes, including a trip worth PLN 10,000 (circa USD 2,800) and five iPads. The airline advertising strategy resulted in 181 complete responses.

Figure 1. Airline advertisement automatically placed on passengers’ self-printed boarding passes

Note: The text in Polish says: ‘Take part in our study and win a trip worth 10,000 zl or one of five iPad mini 16 GB! Learn more about the survey at the project’s page famac-ankieta.com’.

On 21 April 2017, when it became clear that the airline strategy had failed, a targeted Facebook campaign was launched and ran for 30 consecutive days until 20 May 2017. The objective of the campaign was to generate traffic to the external website which hosted the survey. The campaign had a total budget of NOK 48,635 (circa US$ 5,655 during the campaign’s period). The Facebook ad was organised as an automatic auction aiming at the most link clicks at the best price and had a standard delivery type, normally used for automatic bid pricing. The campaign comprised of a single advert which targeted all Polish-speaking Facebook users aged 20–65+ located in Norway, Sweden and the UK at the time of the campaign. The Facebook ad was an 80-second video explaining the goals of the overall research project, with a short text in Polish about the eligibility criteria, prizes and the link to an external website placed above the video (see Figure 2). Once the user clicked on the ad, he or she was automatically taken to an external website where the survey was hosted. The ad was only placed in users’ News Feed and nowhere else (e.g. the right-hand side bar). Due to its dynamic affect, a video ad was deemed to be a more attractive alternative than a picture ad, as it automatically starts playing when it appears on users’ screens. It was run on all types of device, including desktop computers, tablets and mobile phones. No other advertising platform outside of Facebook, such as Instagram or Audience Networks, was used. Due to privacy concerns and to protect respondents’ identity, no tracking (e.g., Facebook Pixel) was used. The Facebook advertising strategy yielded 3,552 complete responses.

Figure 2. Facebook advertisement featured in users’ News Feed

Notes: The text in Polish above the video reads: ‘Are you from Poland and living in Norway, Sweden or Great Britain? Fill out our questionnaire on the topic of migration by latest 20 May (Saturday) and win a trip worth 10,000 zl or an iPad: URL link’. The headline in Polish next to the ‘Learn More’ button beneath the video reads ‘Fill out the survey online here’.

Online survey

The participants recruited to this study were invited to fill out a comprehensive web survey comprising 12 sections with questions ranging from participants’ sociodemographic data and personal migration history to their employment situation and future plans. The survey, which received ethical approval by the Norwegian Centre for Research Data, did not collect participants’ identifiable information. Should the participants have wished to, it was possible to submit answers anonymously without providing an email address. However, the survey did set cookies to prevent repeat participation. To minimise item non-response, most of the questions in the survey were mandatory, with the option ‘Other’ in case respondents found the answer options incomplete or irrelevant for them. Furthermore, backward navigation was enabled in case participants wished to go back and correct the answers they had provided in the previous sections of the survey. Finally, the flexible format of the survey layout was compatible with both mobile and stationary devices. Challenges typically associated with cross-national comparative surveys, such as different understandings of context- and culture-specific concepts among respondents in different countries, were less relevant for this study as its target population comprised of Polish-speaking adults who had emigrated from Poland to Norway, Sweden or the UK. The questionnaire was drafted in English to allow the research-team members to comment and contribute to its development. It was then independently translated into Polish by two native-speaking translators. Asking survey questions in migrants’ native language enhances question comprehension and facilitates the participation of those who would otherwise drop out due to insufficient proficiency in a foreign language. In addition, it eliminates the problem of different question wording in different languages. Finally, it serves as an additional screening tool that prevented non-Polish-speaking persons from participating. Given the responsive design of the survey, its length varied between 40 and 60 questions and it took respondents an average of 15 minutes to complete (median time: 11 min. 50 sec).

Results

The survey instrument registered 6,072 records during the combined period of 50 days in April–May 2017, of which 63 (about 1 per cent) came in between 1 and 20 April when only the initial airline campaign was active. The remaining 6,009 records (99 per cent) came in between 21 April and 20 May 2020, only after the Facebook ad had been launched. Having subtracted the duplicates (n=19) and the entries with information missing on all items (n=414), the survey collected 5,639 complete and incomplete responses. On the last page of the survey, the participants were asked where they had learned about the survey. Among the 3,808 respondents who answered this question (from here on, complete responses), 3,552 (93.3 per cent of all) reported being recruited via the Facebook ad, 181 (4.7 per cent) via the airline ad and 75 (2 per cent) via friends/other. Given the goal of this article, only the results of the Facebook recruitment strategy will be presented and discussed in the following sections.

Facebook Ad’s Target and Actual Reach

At the time of the advertising campaign, the Facebook Ads Manager estimated that the Facebook ad could potentially reach up to 947,000 eligible users: 80,000 in Norway, 67,000 in Sweden and 800,000 in the UK. The Actual Reach in Table 1 shows that the number of unique Facebook users whose News Feed featured the advert during the campaign period was 43,347 in Norway, 40,982 in Sweden and 190,203 in the UK. Thus, the ratio of the Actual and the Potential Reach across the three countries of study ranged from 0.24 in the UK to 0.54 in Norway and 0.61 in Sweden, suggesting a deeper penetration of the ad among its target population in the Scandinavian countries. Over the course of the campaign, the ad was shown on average 3.47 times to 274,532 unique Facebook users who matched the target criteria, resulting in 953,730 impressions and yielding 9,296 unique link clicks (or 3.4 per cent of unique users). Estimating how well the target audience was responding to the advert, Facebook assigned the campaign a relevance score of 8 out of 10, suggesting a high relevance of the advert to its target population. The frequencies shown in Table 1 indicate that the advert was, on average, shown more often to the users located in Norway (4.05 times) and Sweden (5.27 times) than those located in the UK (2.96 times). With regards to the responsiveness of the target audience, it is interesting to see that users in Sweden were more likely to click on the ad (54.95 unique link CTR) than users in Norway (31.84 unique link CTR) or the UK (29.78 unique link CTR).

Table 1. Effectiveness indicators of the Facebook advertising campaign by country

Notes: In advertising, Impressions colloquially refers to the total number of times an ad is displayed to its target audience and is the product of Actual Reach by Frequency. CTR stands for Click-through-Ratio and is calculated as the number of Clicks per 1,000 Impressions.

With 10,618 total link clicks yielded during the Facebook ad campaign, the cost per click can be estimated at NOK 4.58 (or US$ 0.56), although this varied according to participants’ gender (NOK 3.80 for women and NOK 5.33 for men) and across locations (Norway NOK 7.13, Sweden NOK 5.17, the UK NOK 3.66). The Facebook advertising strategy brought in 3,552 complete responses at a total cost of NOK 48,635 (or US$ 5,655), resulting in a price of NOK 13.7 (or US$ 1.6) per complete response. Based on the size of the Facebook-recruited subsample, the Facebook ad response rate can be roughly estimated at 1.3 per cent of the unique Facebook users to whom the advert was shown (n=274,532), a figure which is likely to be underestimated, as it ignores all 1,831 partial responses with missing information on where the respondents learned about the survey, of which an overwhelming majority (n=1,822) came in after the Facebook campaign had been launched. If the share of the Facebook-recruited responses in the total number of complete responses (0.933) is applied to the total number of partial responses which came in when both recruitment strategies were active, a further 1,700 partial responses are likely to have been recruited via the Facebook ad. When both complete and incomplete Facebook-related responses (n=5,252) are factored in, the response rate of the Facebook ad was 1.9 per cent and the price per response was about NOK 9.3 (or US$ 1.1).

Demographic composition of the samples

The composition of the Facebook-recruited samples is presented below with regards to their gender, age and geographical distribution across Norwegian, Swedish and UK regions and, finally, according to respondents’ reported length of residence in their respective countries.

Facebook Ads Manager provides information about the Actual Reach of the ad and the demographics of its audience. Table 2 breaks down the numbers of Actual Reach and Link clicks on the Facebook ad among the different age groups. The vast majority of all clicks on the Facebook ad were performed by users aged 25–44 years (61 per cent of all female and 67 per cent of all male Link clicks), who represent the majority of the Polish adult population in the three countries of study (see discussion below). However, respondents aged 45 years+ were three to four times as likely to click on the ad as users below the age of 45 (see Link CTR figures highlighted in italics). Whereas those aged 18–44 years responded to the ad with 19 to 38 link clicks per each 1,000 reached profiles, users aged 45 years and older produced on average between 61 and 101 link clicks for each 1,000 reached profiles, with certain variations between the genders and age groups.

Table 2. Target population’s responsiveness to the Facebook advert according to gender and age

As for the geographical distribution of the Facebook sample, the majority of the total 3,552 Facebook respondents resided in the UK (n=2,123 or 59.8 per cent), followed by Sweden (n=812, 22.9 per cent) and Norway (n=591, 16.6 per cent). Just a tiny proportion of them (26 cases or 0.7 per cent of all) resided in countries other than the three under study. Figures 3a, 3b, and 3c illustrate how well the Facebook-recruited national samples reflect the official regional distribution of the Polish migrant population in the three countries of study. This is demonstrated by the factor by which the region’s share of the Facebook subsample over- (factor values >1) or underrepresents (factor values <1) the respective region’s actual share of the official Polish migrant population.

Figures 3a, 3b, 3c. Maps illustrating overrepresentation or underrepresentation of regional shares of the Facebook-recruited national samples

Notes: Overrepresentation (factor values > 1) or underrepresentation (factor values < 1) of regional shares of the Facebook-recruited national samples when compared with the official distribution of the Polish migrant population across the Norwegian (3a) and Swedish (3b) counties and the 2016 first-level NUTS regions of the UK (3c). Term ‘migrant population’ refers here to Polish-born persons in Norway and Sweden and Polish citizens in the UK.

Sources: SSB (2021a), SCB (2021c), and ONS (2017c). Maps created in RStudio with the ggmap package (Kahle and Wickham 2013).

Finally, Table 3 presents the composition of the Facebook-recruited national samples according respondents’ reported length of residence in their respective country. With 25.5 per cent of its respondents having reported living in Sweden for more than 13 years, the Swedish sample clearly stands in contrast to the Norwegian and the UK samples.

Table 3. Distribution of national samples according to respondents’ reported length of residence in the immigration country

Discussion

In this section, the accuracy of the language-based targeting strategy is appraised in a twofold manner. Firstly, official statistics on the Polish migrant population are used to evaluate how well the Facebook ad’s Potential Reach covered the target population in the three countries of study. Secondly, the composition of the Facebook-recruited sample is analysed across the lines of migrants’ gender, age, geographical distribution and length of residence in the host country. Since the Facebook advert only targeted users aged 20 years and older, the official statistics presented in what follows are restricted to Polish-born persons aged 20+, if not specified otherwise.

Coverage: Facebook’s Potential Reach vis-à-vis the official immigrant population

The Facebook advertising strategy in this project relied primarily on users’ command of Polish (in both the ad and the survey) and the geographical location of the targeted users in Norway, Sweden or the UK. As of April 2017, Facebook estimated the Potential Reach for the advert to be 947,000 users: 80,000 in Norway, 67,000 in Sweden and 800,000 in the UK. Table 4 confirms that this study’s Potential Reach estimates for the UK and, to large extent, Norway, corresponded well with the official statistics on both the Polish-born and the Polish citizen population but deviated considerably in the Swedish case.

Table 4. Facebook advert’s Potential Reach versus the Polish-born and Polish-citizen population in Norway, Sweden and the UK, 2017

Notes: Norway’s figures are as of 1 January 2017; the citizen figure excludes persons who have naturalised (SSB 2021a, 2021c). Sweden’s figures are as of 31 December 2016; the citizen figure excludes persons with dual citizenship, of which one is Swedish (SCB 2021 b, c). The UK figures are estimates based on ONS (2017c) and the age distributions found in Hawkins and Moses (2016).

As can be seen, the Potential Reach in Sweden was 16 per cent below the number of Polish-born persons and 63 per cent higher than the number of Polish citizens. Migrants’ age and naturalisation patterns are the two likely explanations here. Table 5 provides an overview of the age distribution among Polish-born persons in Norway, Sweden and the UK over 20 years of age. It is noticeable that the proportion of persons aged 65+ was significantly higher in Sweden (14.3 per cent) than in Norway (1.6 per cent) or the UK (2 per cent). Thus, among 79,835 Polish-born persons aged 20+ who resided in Sweden at the beginning of 2017, about 11,440 were 65 years or older. If we consider persons aged 50 and older, their share in the total Polish-born population, aged 20 or above, was 37.5 per cent in Sweden, compared to 16.8 per cent in Norway or just 10.7 per cent in the UK. It is, therefore, reasonable to suggest that, given that older persons are less likely to use social media, the deflated Potential Reach in the Swedish case is due to the relatively higher proportion of older migrants in the Polish-born population in Sweden.

Table 5. Age distribution of Polish-born persons aged 20+ years in Norway, Sweden and the UK, 2017

Notes: Data retrieved from SSB (2021a, 2021c), SCB (2021b, c). Sweden’s figures are as of 31 December 2016, Norway’s as of 1 January 2017. The UK figures are estimates based on ONS (2017a, c) and the age distributions found in Hawkins and Moses (2016).

Further, the discrepancy between the Potential Reach and the Polish citizen population in the Swedish case (see Table 4) deserves further reflection. In general, statistics on foreign citizens are normally less suited to serve as proxies for immigrant population stocks as they tend to distort the numbers both downwards and upwards. On the one hand, when immigrants naturalise, they tend to be excluded from the official statistics on foreign citizens, especially in countries where dual citizenship is either not permitted, not recognised or not registered, to avoid double counting. Table 6 shows the distribution of the Polish-born population aged 20+ years according the nationality they held as of early 2017 in Norway, Sweden and the UK. The Potential Reach for Sweden, being 63.4 per cent larger than the official Polish citizen population of 20+ years old, can be explained by the fact that the foreign citizen statistics exclude persons with dual citizenship, of which one is Swedish.

Table 6. Polish-born persons aged 20+ years in Norway, Sweden and the UK according to the nationality held, 2017

Notes: Sweden’s figures are as of 31 December 2016. Statistics Sweden does not provide information about dual citizenship for Swedish citizens. Norway’s and the UK’s figures are the author’s own estimates based on SSB (2021a, b, c), ONS (2017a, b, c) and the age distributions found in Hawkins and Moses (2016).

On the other hand, statistics on foreign citizens tend to overestimate the migrant population as they also include locally born children who have acquired their immigrant parents’ citizenship at birth but are not immigrants themselves. In the context of this study, the citizen figures in Table 4 are very unlikely to include significantly high numbers of persons born to Polish parents in Norway, Sweden or the UK, since persons under 20 years of age were not targeted by the Facebook ad. For the opposite to be the case, such people must have been born in the respective country of immigration prior to January 1997 and must have acquired Polish citizenship either at birth or later, which is negligible given the low stocks of Polish migrants across the studied contexts at that time. To confirm this line of reasoning, compare the total number of Polish citizens aged 20+ years old in Sweden (41,014 persons in Table 4) with the total number of Polish-born persons aged 20+ years old holding Polish citizenship in Sweden (40,557 persons in Table 6). Thus, only 457 persons out of 41,014 Polish citizens aged 20 and older living in Sweden as of early 2017 were not born in Poland.

Another issue concerning the accuracy of sampling frames is the risk of including persons who are not in the target population. Given that this study’s Facebook recruitment strategy relied primarily on users’ command of the Polish language and their location inferred from their devices’ IP address, any Polish-speaking person who, at the time of the survey, happened to be in one of the three countries of study but did not reside there (i.e. Polish tourists, family members visiting from Poland) could access the survey had they been exposed to the Facebook ad. Given the fact that only 0.7 per cent of all respondents (26 out of 3,552 persons) who had been recruited to the study via the Facebook ad said they resided in countries other than the three of study, it is reasonable to conclude that the Potential Reach figures in this study are unlikely to include significant numbers of persons who did not belong to the target population.

This study’s Potential Reach estimate for the UK, at 800,000 users, was in stark contrast with the estimates reported in studies which relied on another proxy for users’ migration background. Pötzschke and Braun (2017), who narrowed their selection criteria down to ‘Expats (Poland)’ who were ‘away from hometown’, reported Potential Reach figures for the UK of 410,000 users as of November 2016. Ersanilli and van der Gaag (2020), who used a similar strategy except for the ‘away from hometown’, reported 320,000 users as of May 2019 (E. Ersanilli, personal communication, 21 May 2021). Such drastic differences in the Potential Reach figures for the same target population in the same geographical context across the three studies confirm that Potential Reach estimates are sensitive to additional criteria, something that researchers should reflect better on when designing suitable recruitment strategies. Whereas it remains unclear to researchers why the ‘Lived in – country’ (former ‘Expats’) criterion reduces the Potential Reach to such a drastic extent, a viable answer may lie in the algorithms underlying the way(s) in which Facebook infers information about users’ previous residence abroad. Contrary to this latter, there appears to be more clarity with regards to how Facebook infers language(s) understood by its users. Although its claims cannot be verified, a blog post from 2018 explains how Facebook learns about the languages which its users understand. Firstly, it registers both the language of users’ browser and the language of their Facebook interface. Secondly, it considers the languages declared by users in the personal information section, which may be manipulated by the users themselves. Finally and most importantly, Facebook infers the languages its users understand based on their interactions on and off Facebook, such as likes, comments and shares (Devoy 2018). Combined, these methods provide Facebook with much more information about its users’ familiarity with different languages than that which the users themselves declare in their account.

To conclude, the Potential Reach figures based on the language criterion used in this study seem to correspond quite well with the size of the targeted population in Norway and the UK, where the overwhelming majority of the Polish migrants settled after the 2004 EU enlargement. In Sweden, which features a longer history of Polish immigration and where over 14 per cent of the targeted Polish-born population were 65 years or older, the Potential Reach was some 16 per cent smaller that the size of the targeted population. Zooming in on the UK context, the results suggest that the language-based targeting criteria provided this study with a much broader coverage of the target population, compared to studies which relied on the ‘Lived in – Poland’ (formerly ‘Expats Poland’) criterion. However, the language-based targeting of migrants may turn problematic with languages that are spoken beyond the borders of a single state, such as Arabic, English or Spanish (for a discussion of the use of Spanish in targeting Argentinian migrants in Europe, see Ersanilli and van der Gaag 2020). Furthermore, this strategy may not be feasible with a number of languages that are not yet made available in the Facebook Ads Manager, such as Somali, Tigrinya or Amharic.

Demographics of the recruited samples vis-à-vis the official immigrant population

To further appraise the adequacy of language-based targeting, it is of interest to analyse the socio-demographic composition of the recruited samples. In what follows, the Facebook-recruited sample (n=3,552) is analysed according to respondents’ gender, age, geographical distribution and length of residence in each of the studied countries.

At the time that this survey was conducted, in spring 2017, the male-to-female ratio among Polish immigrants aged 20 and above (i.e., the target population) was, respectively, 68 per cent versus 32 per cent in Norway (SSB 2021c), 45 per cent versus 55 per cent in Sweden (SCB 2021c) and 49 per cent versus 51 per cent in the UK (ONS 2017b). Despite the fact that the gender ratios of the Facebook ad’s reach and the link clicks were mostly balanced (see Table 2), women were slightly overrepresented in the overall Facebook-recruited sample (55.5 per cent of all respondents) – 43.7 per cent of the sample in Norway, 59 per cent in Sweden and 57.9 per cent in the UK. It appears that, whereas male and female Facebook users on average did not differ much in their tendencies to click on the Facebook ad (see CTR in Table 2), the female migrants were more likely to take and complete the survey. To better reflect migrants’ gender ratio in a given geographical context, it may be useful to design several ads and allocate proportional budgets separately for men and for women, like Pötzschke and Braun (2017) did in their study. In the case of a single advert targeting both genders, as was done in this study, a possible solution is to temporally restrict the advert’s target audience so that the gender ratio in the recruited sample better reflects the gender ratio in the studied migrant population.

Concerning age, Polish migrants in Norway, Sweden and the UK tend to concentrate in working-age groups and are younger than the overall population in the countries of study (SCB 2021b; SSB 2021c; Hawkins and Moses 2016). As can be seen in Figure 4, the national samples follow very closely the age distribution of the Polish-born population in Norway and the UK. However, older cohorts, aged 55 and above, seem to be underrepresented in some contexts. The bias is especially pronounced in the Swedish and, to some extent, Norwegian samples due to the larger proportion of older migrants in their Polish-born population, compared to the UK. Surprisingly, the opposite is true for the UK sample, where younger cohorts appear to be underrepresented. Despite these variations, it is evident that migrants aged 65 years and older are significantly underrepresented across all samples.

What deserves special attention is the tendency of older Facebook users to engage more actively with the Facebook ad, which holds true for both male and female respondents (see Table 2 in the Results). Such a tendency becomes even more pronounced the older the audience gets, reaching its maximum in the age group 55–64 years old before dropping again for users over 64 years of age. To a certain degree, this tendency could compensate for the fact that the older cohorts of migrants are less represented on Facebook. However, researchers must consider the potential of selection bias for this age group, as those who are found on social media may, in certain aspects such as digital skills, differ considerably from those who are not.

With regards to the geographical distribution of the Facebook sample, it follows very closely the actual distributions of the Polish migrant population in Norwegian (Figure 3a), Swedish (Figure 3b) and UK regions (Figure 3c). Thus, four Norwegian counties (Oslo, Akershus, Rogaland and Hordaland) which, as of January 2017, accounted for about half of all Polish-born persons in Norway, made up 53.5 per cent of the Facebook subsample. In Sweden, three-quarters of all Polish-born residents lived, as of early 2017, in the country’s three most populous regions of Stockholm, Gothenburg and Malmö. These three regions made up 71 per cent of the Facebook subsample in Sweden. In the UK, respondents from all but two regions seemed to be proportionally represented in the Facebook subsample, with the North East being slightly over- and the South East somewhat underrepresented. Facebook’s advantage in targeting users based on their devices’ IP address assured a well-balanced geographical penetration of the recruitment advert, ensuring that even users in the most remote areas could participate in the study.

Figure 4. Age distribution of the Facebook-recruited national samples versus the age distribution of Polish-born persons aged 20–65 in the study countries

Sources: SSB (2021c), SCB (2021c) and Hawkins and Moses (2016).

Finally, given this study’s goal to primarily recruit Polish migrants who settled in their immigration country after the EU enlargement in 2004, it is of interest to look at respondents’ reported length of residence (see Table 3 in the Results). Table 7 below compares Facebook respondents’ reported length of residence versus that found in official sources.

Table 7. Polish-born persons, according to the length of residence in Norway, Sweden, and the UK, 2017

Note: Norway’s official data are for Polish-born persons of all ages (SSB 2021b). Sweden’s official figure is for Polish-born people aged 20 years and above (SCB 2021a). The UK official figures are not available; the <9.6 per cent for those with over 13 years of residence is the author’s own estimate based on total Polish migrant stocks in 2004 and 2017 (ONS 2017c).

As can be seen from Table 7, the pre-accession immigrants – that is to say, those with over 13 years of residence in the country of immigration – mostly concerned the Swedish sample. In Sweden, a relatively large Polish-born diaspora resided well before the 2004 EU enlargement. Between 2004 and 2017, its size merely doubled from 41,608 to 88,704 persons, whereas Norway and the UK saw a remarkable growth of 1,330 per cent and 1,110 per cent respectively (ONS 2017c; SSB 2021a; SCB 2021c). At the same time, the Polish migrant population in Sweden is also, on average, older – which is positively correlated with migrants’ length of residence. The exclusion of the older cohorts of migrants, reflected in the deflated Potential Reach for Sweden (see above), can explain why only 25.5 per cent of the Swedish sample reported over 13 years of residence compared to 39.2 per cent for the official Polish-born population. Despite some differences in the proportions between certain time ranges, it seems that the Facebook ad reached well the earlier and later cohorts of Polish migrants in the three studied contexts.

Conclusions and limitations

This study adds to the emerging body of literature which documents the use of online social networks for recruiting migrants to social research. In line with other studies which targeted migrants using Facebook Ads Manager, the strategy employed in this study proved to be both effective and cost-efficient. Online connectivity wherever and whenever users access their accounts gives Facebook a unique advantage in targeting potential survey respondents. Despite researchers’ rising awareness of this potential, studies which have tapped into this opportunity often fail to assess their Potential Reach or to analyse how representative their recruited samples are. The goal of this article, therefore, was to look beyond the simple metric results and assess the accuracy of a targeting strategy which relied primarily on users’ command of a language as an indicator of their migration background.

Contrary to studies which relied on users’ previous residence abroad, known as the ‘Lived – in’ targeting criterion, this study used migrants’ language (Polish) to target and recruit Polish migrants aged 20 years and older in Norway, Sweden and the UK. Given the criteria used in this study, Facebook estimated the Potential Reach for the advert to be 947,000 users: 80,000 in Norway, 67,000 in Sweden and 800,000 in the UK. By contrasting the Potential Reach figures with the official statistics on the Polish-born population in the three countries of study, this article has shown that targeting based primarily on users’ language (Polish) provided a fairly good coverage across all studied contexts. Thus, the Potential Reach figures corresponded very neatly with the official Polish-born population in Norway and the UK, where most Polish migrants were quite recent arrivals (with less than 13 years of residence) and on average younger. In Sweden, however, where Polish migrants tend to be older and with longer residence abroad, the Potential Reach was 16 per cent lower than the size of country’s Polish-born population. Further, the analysis of the Facebook-recruited sample has shown that oversampling was not an issue, given that very few respondents (0.7 per cent of the sample) were outside the target population. More importantly, this study’s Potential Reach for the UK, estimated at 800,000 users, was clearly larger than the reported figure of 410,000 in Pötzschke and Braun (2017) and 320,000 users in Ersanilli and van der Gaag (2020), two similar studies which, instead, used the ‘Lived in – country’ (formerly known as ‘Expats – country’) criterion in targeting Polish migrants in the UK. In order to significantly increase the numbers of potentially eligible respondents included in Potential Reach, I argue, researchers should consider different targeting strategies, taking into account the demographics of the migrant groups whom they target and the specificity of the languages these migrants speak. This study has shown that targeting strategies based on Facebook users’ language can be effective in cases where migrants’ linguistic, ethnic and national boundaries coincide, as in the case of Polish migrants. However, this may turn out to be problematic for other languages – such as Spanish or Arabic – that transcend ethnic or national boundaries or for those languages that are not available in the Facebook Ads Manager. As was the case for other studies in this emerging field of research, the key limitation of this study was related to the algorithms used in Facebook Ads. As the mechanisms underlying the classification of users into certain categories remain sometimes opaque, migration scholars should be cautious when designing targeting strategies and compare their Potential Reach with the data from official sources.

The analysis of the socio-demographic composition of the collected samples has highlighted the importance of considering both the socio-demographic composition of the targeted migrant groups and the history of their migration in each studied context. Thanks to the Internet’s penetration to even the most remote areas of Norway, Sweden and the UK, the regional distribution of the Facebook subsample in this study accurately followed the actual regional distribution of the Polish migrants in the three countries. Concerning gender balance, although male respondents recruited via the Facebook advert seemed to be less likely than women to participate, this could be mitigated by either temporally restricting the advert’s target audience to male respondents only or by setting up advert sets and budgets separately for men and women. When it comes to the age of respondents, this study has shown that, in countries where access to the internet is almost universal, even presumably hard-to-reach social groups, such as older male migrants, may show a greater interest in participating in survey research than younger users. In this study, Facebook users aged 45–64 years old were between three and four times more likely to click on the ad than users below the age of 45. This should sound particularly promising to researchers aiming to use Facebook for recruiting older migrants to their research. However, older cohorts of migrants, especially those above 65 years of age, were underrepresented in the national subsamples. In analysis, quantitative data collected via online social networks must be weighted to better reflect the socio-demographic composition of the studied migrant population.

Note

¹ Soon after, Facebook replaced the ‘Expats (country)’ criterion by the ‘Lived in – country’ category although the logic remained the same.

Availability of data and materials

The original data analysed during the current study can be made available from the corresponding author on reasonable request.

Acknowledgements

I would like to thank my two PhD supervisors and my PhD fellow Mats Eirik Lillehagen at the University of Oslo for their critical comments and valuable advice during this manuscript preparation, as well as the two anonymous peer reviewers for their constructive and helpful critique.

Funding

This study was made possible thanks to a grant from the Research Council of Norway (grant no. 250638). The funding body was not involved in the design of the study or the collection, analysis, or interpretation of data.

Conflict of interest statement

No conflict of interest was reported by the author.

ORCID ID

Oleksandr Ryndyk  https://orcid.org/0000-0001-6590-544X

References

Cassar C. M., Gauci J. P., Bacchi A. (2016). Migrants’ Use of Social Media in Malta. The People for Change Foundation. Online: http://www.pfcmalta.org/migrants-use-of-social-media.html (accessed: 8 December 2021).

Curtin R., Presser S., Singer E. (2005). Changes in Telephone Survey Nonresponse over the Past Quarter Century. Public Opinion Quarterly 69(1): 87–98.

de Leeuw E. D., de Heer W. (2002). Trends in Household Survey Nonresponse: A Longitudinal and International Comparison, in: R. M. Groves, D. A. Dillman, J. L. Eltinge, R. J. A. Little (eds), Survey Nonresponse, pp. 41–54. New York: Wiley.

Devoy F. (2018). The Mystery of Facebook Language Targeting. Adviso Blog, 26 January. Online: https://www.adviso.ca/en/blog/tech-en/mystery-facebook-language-targeting/ (accessed: 8 December 2021).

Dubois A., Zagheni E., Garimella K., Weber I. (2018). Studying Migrant Assimilation through Facebook Interests, in: S. Staab, O. Koltsova, D. Ignatov (eds), Social Informatics. SocInfo 2018. Lecture Notes in Computer Science, pp. 51–60. Cham: Springer.

Eisner M., Ribeaud D. (2007). Conducting a Criminological Survey in a Culturally Diverse Context: Lessons from the Zurich Project on the Social Development of Children. European Journal of Criminology 4(3): 271–298.

Ersanilli E., van der Gaag M. (2020). MOBILISE Data Report: Online Surveys. Wave 1. MOBILISE Working papers.

Facebook (2021). Facebook Reports First Quarter 2021 Results. Press Release 28 April 2021. Online: https://investor.fb.com/investor-news/press-release-details/2021/Faceboo... (accessed: 8 December 2021).

Feskens R. C. W. (2009). Difficult Groups in Survey Research and the Development of Tailor-Made Approach Strategies. Utrecht: Utrecht University, Centraal Bureau voor de Statistiek.

Feskens R. C. W., Hox J. J., Lensvelt-Mulders G. J. L. M., Schmeets J. J. G. (2006). Collecting Data among Ethnic Minorities in an International Perspective. Field Methods 18(3): 284–304.

Feskens R. C. W., Kappelhof J., Dagevos J., Stoop I. A. L. (2010). Minderheden in de mixed-mode? Een inventarisatie van voor- en nadelen van het inzetten van verschillende dataverzamelingsmethoden onder niet-westerse migranten. The Hague: SCP.

Hawkins O., Moses A. (2016). Polish Population of the United Kingdom. Briefing Paper Number CBP7660. London: House of Commons Library.

Hunter A. (2015). Empowering or Impeding Return Migration? ICT, Mobile Phones, and Older Migrants’ Communications with Home. Global Networks 15(4): 485–502.

Iannelli L., Giglietto F., Rossi L., Zurovac E. (2018). Facebook Digital Traces for Survey Research: Assessing the Efficiency and Effectiveness of a Facebook Ad-Based Procedure for Recruiting Online Survey Respondents in Niche and Difficult-to-Reach Populations. Social Science Computer Review 38(4): 462–476.

Internet World Stats (2021). The Internet Big Picture: World Internet Users and 2021 Population Stats. Online: http://www.internetworldstats.com/stats.htm (accessed 8 December 2021).

Kahle D., Wickham H. (2013). ggmap: Spatial Visualization with ggplot2. The R Journal 5(1): 144–161.

Kapp J. M., Peters C., Oliver D. P. (2013). Research Recruitment Using Facebook Advertising: Big Potential, Big Challenges. Journal of Cancer Education 28(1): 134–137.

Kappelhof J. (2014). The Effect of Different Survey Designs on Nonresponse in Surveys among Non-Western Minorities in the Netherlands. Survey Research Methods 8(2): 81–98.

Kayrouz R., Dear B. F., Karin E., Titov N. (2016). Facebook as an Effective Recruitment Strategy for Mental Health Research of Hard-to-Reach Populations. Internet Interventions 4: 1–10.

King-O’Riain R. C. (2015). in Transnational Families in Ireland. Global Networks 15(2): 256–273.

ONS (2017b). Annual Population Survey – Regional – Nationality. Year Ending March 2017. Dataset. Nomis – Official Labour Market Statistics. London: Office for National Statistics. Online: https://www.nomisweb.co.uk/query/select/getdatasetbytheme.asp (accessed: 9 December 2021).

ONS (2017a). Annual Population Survey – Regional – Country of Birth. Year Ending March 2017. Dataset. Nomis – Official Labour Market Statistics. London: Office for National Statistics. Online: https://www.nomisweb.co.uk/query/select/getdatasetbytheme.asp (accessed: 9 December 2021).

ONS (2017c). Population of the UK by Country of Birth and Nationality. Year Ending June 2017. London: Office for National Statistics. Online: https://www.ons.gov.uk/  (accessed: 8 December 2021).

Palotti J., Adler N., Morales-Guzman A., Villaveces J., Sekara V., Garcia Herranz M., Al Asad M., Weber I. (2020). Monitoring of the Venezuelan Exodus through Facebook’s Advertising Platform. PLoS ONE 15(2): e0229175.

Pedersen E. R., Kurz J. (2016). Using Facebook for Health-Related Research Study Recruitment and Program Delivery. Current Opinion in Psychology 9: 38–43.

Pötzschke S., Braun M. (2017). Migrant Sampling Using Facebook Advertisements: A Case Study of Polish Migrants in Four European Countries. Social Science Computer Review 35(5): 633–653.

Ramo D. E., Prochaska J. J. (2012). Broad Reach and Targeted Recruitment Using Facebook for an Online Survey of Young Adult Substance Use. Journal of Medical Internet Research 14(1): e28.

Rife S. C., Cate K. L., Kosinski M., Stillwell D. (2016). Participant Recruitment and Data Collection through Facebook: The Role of Personality Factors. International Journal of Social Research Methodology 19(1): 69–83.

SCB (2021a). Foreign-Born by Country of Birth and Year since Last Immigration. Year 2000–2020. Stockholm: Statistics Sweden. Online: https://www.statistikdatabasen.scb.se/pxweb/en/ssd/ (accessed: 9 December 2021).

SCB (2021b). Foreign Citizens by Country of Citizenship, Age and Sex. Year 1973–2017. Stockholm: Statistics Sweden. Online: http://www.statistikdatabasen.scb.se/pxweb/en/ssd/ (accessed: 9 December 2021).

SCB (2021c). Population by Country of Birth, Age and Sex. Year 2000–2018. Stockholm: Statistics Sweden. Online: http://www.statistikdatabasen.scb.se/pxweb/en/ssd/ (accessed: 9 December 2021).

Sheehan K. B. (2006). E-mail Survey Response Rates: A Review. Journal of Computer-Mediated Communication 6(2). Online: https://doi.org/10.1111/j.1083-6101.2001.tb00117.x (accessed: 9 December 2021).

Stoop I. A. (2005). The Hunt for the Last Respondent: Nonresponse in Sample Surveys. The Hague: Social and Cultural Planning Office of the Netherlands.

SSB (2021a). Immigrants and Norwegian-Born to Immigrant Parents. Table 05184: Immigrants, by Sex and Country Background 1970–2018. Oslo: Statistics Norway. Online: http://www.ssb.no/en/statbank/table/05184 (accessed: 9 December 2021).

SSB (2021b). Immigrants and Norwegian-Born to Immigrant Parents. Table 10598: Immigrants, by Country Background and Duration of Stay (F) 2005–2018. Oslo: Statistics Norway. Online: https://www.ssb.no/en/statbank/table/10598 (accessed: 9 December 2021).

SSB (2021c). Population. Table 05196: Population, by Sex, Age and Citizenship 1977–2018. Oslo: Statistics Norway. Online: http://www.ssb.no/en/statbank/table/05196/ (accessed: 9 December 2021).

Thornton L., Batterham P. J., Fassnacht D. B., Kay-Lambkin F., Calear A. L., Hunt S. (2016). Recruiting for Health, Medical or Psychosocial Research Using Facebook: Systematic Review. Internet Interventions 4(1): 72–81.

Whitaker C., Stevelink S., Fear N. (2017). The Use of Facebook in Recruiting Participants for Health Research Purposes: A Systematic Review. Journal of Medical Internet Research 19(8): e290.

Yuan P., Bare M. G., Johnson M. O., Saberi P. (2014). Using Online Social Media for Recruitment of Human Immunodeficiency Virus-Positive Participants: A Cross-Sectional Survey. Journal of Medical Internet Research 16(5): e117.

Zagheni E., Weber I., Gummadi K. (2017). Leveraging Facebook’s Advertising Platform to Monitor Stocks of Migrants. Population and Development Review 43(4): 721–727.

Copyright information

© The Author(s)

Open Access

This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.