Sampling Techniques

Sampling techniques are widely used, especially in surveys, but their concepts apply to various fields, including experiments. Take, for example, a study on breast cancer self-examination that required gathering large samples of women from different backgrounds.

At the core of sampling theory is understanding the distinction between a population and a sample. Consider a scenario where you’re estimating the grades of university students worldwide. Initially, your population encompasses all university students globally. If your data collection is limited to South Africa, your population narrows down to university students in South Africa. Further focusing on just one university, such as Wits, makes that university your population. The ability to generalize your findings to all South African students hinges on the resemblance of Wits students to others in the country. Ideally, to draw more accurate conclusions, you’d need samples from all universities in South Africa.

The population is essentially defined by the question being asked and includes all units you aim to draw conclusions about. For instance, if you’re studying families or litters, your sampling unit would be the family or litter, rather than individual members.

In reality, examining every subject in a population isn’t always feasible due to constraints like cost, time, or the destructive nature of testing. Therefore, researchers often focus on a subset of the population. This approach allows for more thorough data collection and ensures representation of hard-to-reach units, such as remote villages or specific animal habitats.

Sampling was crucial during the 1990 South African population census, especially for gathering data on squatter camps. These camps lacked conventional streets or fixed addresses, making it challenging to ensure that each dwelling was accounted for in the census. Aerial photographs were utilized to identify different types of dwellings, like multiple-family or single-family units, and hostels. By counting the number of each type of dwelling, a sample could be drawn using the aerial photos. Skilled interviewers could then visit selected dwellings to interview occupants. Similar methods were employed in subsequent censuses.

To ensure that conclusions drawn from the sample could be applied to the entire population, it’s crucial for the sample to be representative. This is often achieved by randomly selecting samples from the population, which also helps to avoid personal bias in selection.

It’s important to understand that the representativeness of a sample isn’t determined by its size but rather by how it’s chosen. For instance, selecting the “best” 50 students from various classes doesn’t make the sample more representative than choosing the best 10 or 100 students. However, drawing 20 students randomly (where each has an equal chance of being selected) could indeed be representative. It’s about reducing bias rather than focusing on the number.

Random samples are those drawn in a way that gives each unit of the population an equal chance of being chosen. This allows statistical probability theory to be applied in estimating distributions of statistics, hypothesis testing, and other analyses.

Drawing a random Sample

One of the most straightforward methods for drawing a random sample involves numbering every element within the population, writing these numbers on separate pieces of paper, placing them into a hat, thoroughly mixing them, and then randomly selecting the desired number of pieces of paper. However, this approach requires that all members of the population can be identified and listed, forming what’s known as a sampling frame, which essentially defines the population being studied.

An alternative to the hat method is to obtain random numbers from tables or a computer. This approach is preferred because it eliminates the challenge of ensuring thorough mixing of the pieces of paper. To illustrate this challenge, consider the drafting process during the Vietnam War. In an attempt to ensure fairness, birthdates were written on balls and placed into a container for mixing. However, the mixing process was not thorough enough. Balls with birthdates added later were less likely to be drawn, leading to a disproportionate number of individuals drafted from certain months, such as December. This highlights the importance of thorough mixing to achieve true randomness in the sampling process.

Drawing Numbers from Random Number Tables

Drawing numbers from random number tables is another effective method for obtaining a random sample. Here’s how it works:

Open the random number table to any page, and randomly select a starting point.
Assume our population is numbered from 1 to 80, and we want to select 6 numbers.
Starting from the randomly chosen point, read off pairs of numbers.
If the pair represents a number within our population range (1-80), include it in the sample. Otherwise, discard it.
Continue this process until you have the desired number of elements for your sample.
If duplicates are encountered, discard them.

For instance, let’s say we start from the fourth row and the seventh column:

87 49 32 16 20 71 52 10 35 68
45 03 98 64 22 53 77 19 60 09
12 26 81 37 50 05 63 74 28 91
16 88 41 69 57 23 38 48 13 29

Starting from our random point, we read off pairs of numbers: 52, 10, 35, 68, 45, 03, and so on. We continue until we have 6 valid numbers for our sample.

This method ensures randomness in selecting elements from the population and can be easily adjusted for different population sizes and sample requirements.

Obtaining random numbers from a computer

Another method for obtaining a random sample is by using a “uniform” random number generator. Here’s how it works:

Start by generating the required number of random numbers from a uniform [0,1] distribution. These numbers will fall between 0 and 1.
Suppose the population has elements. Multiply each of these random numbers by $(n - 1)$ , and then add 1 to the result. This ensures the numbers will lie between 1 and .
Round off the resulting numbers to obtain integers.

For example, let’s generate 5 random numbers between 0 and 1:

0.699152, 0.413815, 0.079409, 0.793702, 0.370974

Multiplying these numbers by $(59)$ (since we want numbers between 1 and 60) gives:

41.249968, 24.415085, 4.685131, 46.828418, 21.887466

Adding 1 to each

42.249968, 25.415085, 5.685131, 47.828418, 22.887466

Generating Random Numbers from a Uniform Distribution

Another method for obtaining a random sample is by using a “uniform” random number generator. Here’s how it works:

Start by generating the required number of random numbers from a uniform [0,1] distribution. These numbers will fall between 0 and 1.
Suppose the population has $n elements. Multiply each of these random numbers by$ $(n - 1)$
$n$ , and then add 1 to the result. This ensures the numbers will lie between 1 and
$n$
Round off the resulting numbers to obtain integers.

For example, let’s generate 5 random numbers between 0 and 1:

0.699152, 0.413815, 0.079409, 0.793702, 0.370974

Multiplying these numbers by

$(59)$

$(59)$ (since we want numbers between 1 and 60) gives:

41.249968, 24.415085, 4.685131, 46.828418, 21.887466

Adding 1 to each:

42.249968, 25.415085, 5.685131, 47.828418, 22.887466

Rounding off

42, 25, 6, 48, and 23

These rounded numbers represent our random sample from the population. This method ensures randomness and allows for easy adjustment for different population sizes and sample requirements

Sampling with and without replacement

In sampling theory, the concept of “sampling with replacement” is commonly discussed, where numbers can be drawn more than once. Imagine drawing numbers from a hat, putting each number back, and remixing before drawing another number. However, in practice, it’s usually not desirable to include a number more than once. For example, asking the same questions to or examining a person multiple times is unlikely to yield useful information. Therefore, “sampling without replacement” is often preferred.

The formulas for standard errors of estimates are slightly more complex in sampling without replacement. For detailed explanations, you can refer to works such as Kish (1965) or Cochran (1977). Interestingly, for large populations, the formulas for standard errors for both sampling with and without replacement become equivalent.

Finite vs. Infinite Populations

Another technical aspect in sampling theory is the distinction between “finite” and “infinite” populations. In many cases, even if the population is discrete and countable, it’s large enough to be treated as infinite. However, if the population size is not sufficiently large, adjustments are made in the formulas to account for finite populations. This adjustment is known as the “finite population correction,” represented by

$(1 - \frac{n}{N})$

$(1 - N n)$ , where $n$ is the number of units in the sample, and N is the total number of units in the population. This correction ensures more accurate estimates when dealing with smaller populations.

Types of Sampling

When every unit in the population has an equal chance of being selected, the resulting sample is likely to be representative of the entire population. This method is known as random sampling. It stands in contrast to non-random sampling techniques, such as:

Convenience Sampling: In convenience sampling, researchers select the sample based on what is convenient or readily available. This method is often used for its simplicity and accessibility, but it may not accurately represent the entire population.
Quota Sampling: Quota sampling involves selecting individuals based on predetermined quotas or characteristics, such as age, gender, or socioeconomic status. While it ensures representation of specific subgroups within the population, it may introduce bias if the quotas are not properly determined.

Each sampling method has its own strengths and limitations, and researchers choose the most appropriate method based on their study objectives, resources, and constraints.

Non-random sampling

Convenience Sampling: As the name suggests, convenience sampling involves selecting a sample that is easily accessible or convenient. Examples include:

Inspecting only the oranges from the top layer of boxes, which may not be representative of all oranges in the shipment.
Sampling potatoes from the top sack in a stack, potentially missing variations in quality or freshness among different sacks.
Examining plants or animals near a road, which may not accurately represent the overall population due to environmental differences.

While convenience sampling can sometimes yield useful results, it carries the risk of producing non-representative samples. Careful consideration should be given to potential biases before using this method.

Quota Sampling: Quota sampling is often used in opinion surveys, where interviewers are instructed to collect opinions from a specific number of individuals with certain characteristics, such as age or gender. However, this method is fraught with challenges:

Interviewers may struggle to meet their quotas and may select biased individuals or even enlist friends or relatives to participate.
Interviewers may inadvertently select individuals who are similar to each other, leading to a sample that does not accurately represent the population.

While convenience and quota sampling can sometimes provide useful insights, they are associated with more potential biases compared to other sampling methods.

Snowball Sampling: This method is useful when there is no readily available list of the population, and it is difficult to locate members. Researchers begin with a few respondents and then ask them to provide contact details for others. However, researchers must be cautious to avoid biases, such as only reaching individuals with certain characteristics.

Pseudo-Random Sampling: In cases where obtaining a list of the population is impossible, researchers may resort to pseudo-random methods, such as sampling individuals with a specific disease who arrive at a hospital. However, careful consideration is needed to determine whether such a sample accurately represents the population of interest.

Random Sampling

Random Sampling (Probability Sampling): In random sampling, every unit in the population has a known and equal chance of being selected. However, practical constraints may arise, such as the high cost of obtaining observations from every unit in a large population like South Africa. To address this, multistage sampling techniques can be employed. This involves first randomly selecting towns or areas, followed by randomly selecting individuals or animals within each selected town or area.

Stratified Sampling: To ensure representation from all important groups, populations can be divided into strata based on certain characteristics, such as size or socioeconomic status. Random samples are then drawn from each stratum. For example, in a medical study, samples could be drawn from different hospitals and clinics to account for variations in patient demographics.

Proportional to Size Sampling: In some cases, it may be beneficial to draw a sample proportional to the size of each stratum. This ensures a “self-weighting” sample, where each stratum contributes proportionally to the overall estimate. However, careful consideration is needed when stratum sizes vary significantly.

Cluster Sampling: Cluster sampling involves selecting clusters of units at random and including all units within the selected clusters in the sample. This method reduces costs but may introduce correlation, as units within the same cluster are often more similar to each other. For example, in market research, clusters of households may be selected, and individuals from different households within each cluster may be interviewed.

Systematic Sampling: Systematic sampling involves selecting units at regular intervals from a list or sequence. This method is useful when sampling units are listed in some spatial or sequential order. For instance, one could select every nth house on a street or every nth share from a list of share prices.

Each sampling method has its own advantages and limitations, and researchers must carefully consider which method is most appropriate for their study objectives and constraints.

Ecological Sampling Strategies

In ecological studies, sampling strategies often align with broader classifications used in other fields, such as stratified, cluster, or systematic sampling. For instance, quadrat sampling, a common method in ecology, can be viewed as a form of stratified sampling.

Quadrat Sampling: In quadrat sampling, the sampling unit is defined as a quadrat, and the study area is divided into strata or belts. Each belt is sampled by selecting all quadrats within it, resulting in a contiguous set of quadrats being selected, known as the belt-quadrat method. However, a challenge in this method is defining the appropriate quadrat size. Selecting a size too large may obscure the true distribution of populations, making clumped populations appear uniformly distributed.

Challenges in Defining Sampling Units: Defining the sampling unit is critical in ecological sampling, especially in continuous environments like fields, lakes, or forests. The choice of sampling unit, whether it’s a person, house, tree, leaf, etc., can significantly impact the results. For instance, in quadrat sampling, the size of the quadrat must accurately capture the distribution patterns of the species under study.

Ecological sampling methods often require careful consideration of the study environment and the characteristics of the species being sampled to ensure accurate representation and meaningful results.

Weighting in Survey Sampling

In survey sampling, weighting is essential when dealing with probability samples to ensure accurate representation of the population. Here’s how weighting works:

1. Correction for Disproportionate Allocation:
When sample sizes differ across strata due to deliberate boosting in certain areas, weights are assigned to correct for this discrepancy. For instance, if a larger sample is drawn from metropolitan areas compared to rural areas, weights are calculated as the population size divided by the sample size within each area. This correction ensures that estimates are not biased towards overrepresented areas.

2. Post-Stratification Weighting:
Certain demographic groups may be more likely to participate in a survey. For example, elderly women may be more available for interviews than young men. To address this, the realized sample is weighted to match the population distribution across relevant factors such as age, gender, and other key variables of interest. These weights ensure that survey results accurately reflect the population characteristics.

Minimum Sample Size for Reporting:
It’s generally recommended to have a minimum of 40 respondents per reporting category to ensure robust estimates. However, in some cases, this threshold can be lowered to 20 respondents per category.

Other Considerations:
Weighting surveys involves several considerations, including potential biases and adjustments to account for them. Further insights into weighting techniques can be found in literature such as Kish’s work.

Weighting is a crucial step in survey analysis, ensuring that results are representative and reliable, even when sample sizes vary across different segments of the population.

Bias

Sampling strives for unbiased results, meaning they accurately reflect the population. However, bias can sneak in various ways. For instance, if the sampling frame isn’t representative—like using a phone directory to survey a city—certain groups, like the less affluent, may be underrepresented. Similarly, drawing animals from a specific abattoir might not capture the diversity of the entire population if that abattoir mainly serves certain regions, like Limpopo.

In the 1936 American elections, both the Literary Digest and Gallup conducted polls to predict the winner. The Literary Digest surveyed 237,600 people and forecasted a victory for Landon. However, Gallup’s poll of 50,000 individuals predicted Roosevelt’s win, with 56% of the votes. Ultimately, Roosevelt won with 62% of the votes. The key difference was that Gallup used a more representative sample, while the Literary Digest’s sample was biased towards wealthier individuals, as it relied on the telephone directory and car registration lists. This bias favored Roosevelt, who had strong support among lower-income groups.

Non-reports can introduce bias into sampling. For instance:

In ecological studies, inaccessible regions in mountainous areas may result in missing species counts, leading to incomplete data.
In healthcare, when requesting specific patient files, some may be signed out or unavailable due to ongoing treatments or other research projects, skewing the sample.
Patients not returning for follow-up appointments can pose challenges as their outcomes remain unknown, impacting the accuracy of longitudinal studies.
In animal research, illness, fights, or escapes can cause missing data points, affecting the integrity of the study.
Financial analyses may encounter days with unrecorded share prices, leading to incomplete market data and potential misinterpretations.
In market research, individuals declining participation can lead to a biased sample, impacting the generalizability of findings.
Voluntary responses, like those in mail surveys, often come from individuals with strong opinions, leading to bias. Follow-up studies are essential to compare the characteristics of non-respondents with those who respond voluntarily. For instance, a phone-in survey asking if participants would marry the same person again resulted in a 90% “no” response, indicating strong sentiments among respondents. However, this may not represent the broader population’s views accurately.
Volunteers in experiments often differ from non-volunteers, and caution is needed when drawing conclusions from their participation. For instance, estimating the number of HIV-positive individuals through volunteer testing may overestimate the prevalence, as volunteers are likely from higher-risk groups. Similarly, randomly sampling patients at a hospital or clinic may introduce bias, as they are typically sicker than the general population.
Self-selected samples often introduce bias, which can be challenging to address fully. However, efforts can be made to estimate the bias by comparing demographic data from the sample to the overall population or other studies. External validators, such as the number of MNET decoders in use or access to utilities like telephones and electricity, are often employed in market research to assess the generalizability of the findings.
Another source of bias can stem from biased selection by interviewers, even in non-quota samples. This bias may arise from curiosity or other factors. Similarly, biases can emerge in the screening of participants for experiments or the definition of sampling sites or habitats, especially as the study progresses and criteria are adjusted.
Procedural definitions can also introduce bias. For instance, responses may differ depending on whether someone else is present during questioning, as seen in attitudes towards skipping school in the presence of a teacher or parent versus alone with the child. Additionally, measurements on animals or people may vary based on the stressfulness of the environment. It’s essential to standardize the background against which measurements are taken to minimize bias.

Accuracy and Precision

Precision refers to the consistency or reproducibility of an estimate, indicated by a small variance. When estimates are precise, the values of the measured variable are tightly clustered upon repeated sampling. For example, weighing packets of sugar typically yields precise estimates due to careful filling, whereas estimating the average length of wood for fencing may result in imprecise estimates due to variations in cutting.

Accuracy, or lack of bias, pertains to how closely an estimate aligns with the true population parameter. Bias occurs when the sample statistic consistently deviates from the population parameter in the same direction. For instance, repeatedly estimating the number of people with AIDS using self-selected samples may lead to a consistent overestimation.

Ideally, the best estimator is both unbiased (accurate) and precise. However, if achieving this balance is not possible, knowing the size and direction of the bias can help in selecting an estimator. For instance, if an instrument consistently overreads by 5 units but provides precise results, it can be preferable if the bias can be reliably corrected.

Sample Protocol

When beginning an experiment that involves sampling, it’s crucial to establish a sampling protocol, which consists of rules for selecting the sample. This protocol should outline specific criteria for inclusion in the sample and the methodology for identifying potential sample members.

It’s essential to address various situations in the sample selection process, such as encountering “not at home” situations. Simply substituting the next-door neighbor may introduce bias, particularly if certain groups like sales representatives or working women are disproportionately affected. To mitigate bias, substitution should only occur after repeated attempts to contact the originally chosen address or site at different times of the day.

Selecting the date and time for sampling is also critical. When planning a population census, careful consideration is given to avoid public holidays, school holidays, or religious holidays for any relevant religious groups. Choosing summer months is often preferred as it allows for longer visiting hours, considering the tendency for people to avoid answering the door after dark due to security concerns. Similarly, for surveys involving students, it’s best to avoid religious holidays, exam periods, and vacations. When surveying animals or plants, factors such as the condition of the environment and the likelihood of heavy rain should be taken into account.

Pilot Surveys

Pilot surveys are crucial, whether you’re surveying people, habitats, or animals. They involve drawing a small sample from the population you plan to survey and testing your protocol and questionnaire on this group. This allows you to ensure the procedure works smoothly—can you find the desired sample easily, how long does it take to administer the questionnaire, are the questions clear, do any questions cause discomfort, can you easily interpret responses, etc.

For surveys involving plants or animals, you won’t have a questionnaire but likely a checklist or method for noting behaviors. A pilot survey helps refine your checklist, identify problems with definitions, available space on forms, time constraints, simultaneous measurement feasibility, etc. It’s crucial to design the questionnaire and sampling protocol carefully and test them thoroughly since you usually can’t go back to correct mistakes afterward. No amount of statistical analysis can salvage a flawed sample or uninformative data.

You may need multiple pilot surveys to perfect your questionnaire. Include details of how you plan to analyze the data in your study plan. Trying out data entry and analysis on pilot survey results can be helpful. While you won’t draw conclusions due to the small sample size, it’s essential to ensure all aspects of the study are in order to avoid investing time and money only to find the results can’t be analyzed effectively.

Reliability and Validity

Various methods exist for evaluating a questionnaire, many of which stem from psychological testing practices. Anastasi et al. (1997) provide extensive information in this area.

The primary concerns when evaluating a questionnaire are its validity and reliability. Validity refers to the extent to which the questionnaire measures what it intends to measure, while reliability assesses the consistency of responses over different occasions. While they’re related, they’re not the same—something can be reliable without being valid and vice versa. Ideally, a questionnaire should be both reliable and valid.

Common measures of internal consistency include Cronbach’s alpha and the Kuder-Richardson coefficient, which are detailed in section 12.4.

Validity encompasses several components. Content validity assesses whether the questionnaire covers a representative sample of the behavior domain it aims to measure. Face validity gauges whether the questionnaire appears to test what it claims to. If the test seems irrelevant to participants, they may not respond cooperatively. Criterion-related validity assesses how effectively the test predicts outcomes, such as job performance. Predictive validity can be assessed by comparing questionnaire results with later outcomes, while concurrent validity examines how well the questionnaire reflects the current situation. Construct validity evaluates whether the test measures the intended theoretical construct. This can involve correlating questions with each other to assess convergent and discriminant validity or using factor analysis to identify underlying constructs within the data.

Summary of Sample Size Determination

In this section, we delved into the intricacies of sampling methods and their potential biases in ecological research. We explored various sampling strategies, including stratified sampling and quadrat sampling, highlighting the importance of defining the sampling unit accurately to avoid biases. Weighting in probability sampling was discussed as a method to correct for disproportionate allocation in different strata.

The discussion then shifted to bias in sampling, emphasizing the importance of obtaining unbiased results representative of the population. Various sources of bias were identified, such as non-representative sampling frames, non-reports, and voluntary responses. Examples from different fields, including historical instances like the 1936 American elections, illustrated how bias can skew results.

Sampling protocols were introduced as essential guidelines for drawing samples, emphasizing the need for careful consideration of factors like timing and substitution protocols. Pilot surveys were highlighted as crucial for testing methodologies and questionnaires before full-scale implementation, ensuring the validity and reliability of data collection methods.

Finally, the section concluded with a discussion on evaluating questionnaires, focusing on validity and reliability. Different types of validity, including content validity, face validity, criterion-related validity, and construct validity, were explained, along with methods for assessing reliability. Overall, the section provided a comprehensive overview of sampling techniques, bias mitigation strategies, and questionnaire evaluation methods in ecological research.