Methods

From Research Design to Methods: Explaining the How and Why

Bogdan G. Popescu
bogdan.popescu@johncabot.edu

John Cabot University

Structure of Today’s Session

  1. What is a Methods Section?

  2. Research Design

  3. Research Methods

  4. Types of Research Designs

  5. Applying to a Model Article

The Methods Section of the Article

What to include

The methods section should include answers to the following questions:

  • How did you do your research?
  • What was the research design?
  • Was it quantitative or qualitative?

Introduction

The Research Design

The Research Design = structure/framework (e.g., case study, experiment)

It forms the backbone of the methods (tools) sections of a paper.

  • What data did you use and how was it collected?
  • What are your key variables and how were they measured?
  • What analytical methods did you apply?

A good research design tries to ensure high internal and external validity

Introduction

The Research Design

Design = framework/structure/plan

Example: Case study, experiment

Methods = tools/techniques

Example: Interviews, quantitative analysis

The Research Design

Internal Validity

Internal Validity - the extent to which we can be confident that the independent (causal) variable produced the observed effect.

  • Is the independent variable responsible for variation in the dependent variable?
  • What other possible causes might there be for the relationship between the variables?
  • Could something else have been responsible for the variation in the dependent variable?
  • Could there be confounding factors?

The Research Design

External Validity

External Validity - the extent to which the results from a study can be generalized beyond the particular study

  • Can you generalize your findings?
  • Are your conclusions likely to apply more widely?

Methods

Research methods = techniques for collecting and analyzing data (e.g., interviews, regression)

Approaches:

  • Quantitative approaches - usually good at making generalizations: large-N
  • Qualitative approaches - good at investigating these hard-to-define concepts and hard-to-reach populations: small-N
  • Mixed Methods - use both Quantitative and Qualitative approaches

Data sources include:

  • already existing quantitative data
  • experiments
  • surveys
  • interviews or focus groups
  • comparative research
  • archival data and documentary records (e.g., speeches, policy documents)

Units of analysis (observations) can include people, countries, organizations, texts, etc.

Types of Research Designs

Focus of This Course

The ones that we will focus on are:

  1. Comparative Analysis - qualitative
  2. Experimental Designs - quantitative
  3. Quasi-natural Experiments - quantitative

Other research designs that are not presented in this course include: Ethnography/Participant Observation, and Content Analysis, etc.

Comparative Analysis

Case Study: Intro

A case study is a type of comparative analysis, especially when situated within a broader theoretical framework. Small-N comparison can involve multiple case studies.

A case study focuses on the analysis on one country, event, or organization.

It must be situated comparatively to matter beyond itself.

Case studies:

  • apply existing theory to new contexts;
  • examine exceptions to the rule
  • generate new theory
  • have to explain how they are applicable to other contexts
  • explore a causal process or mechanisms (process tracing)

Comparative Analysis

Case Study: Data

Case studies may use

  • interviews
  • surveys
  • ethnography
  • focus groups
  • documents: policy documents and speeches

Comparative Analysis

Case Study: Examples

Voting behavior in Britain

Public Attitudes towards the environment in Germany

Public attitudes towards immigrants and ethnic minorities in the Netherlands.

Comparative Analysis

Small-N Comparative Designs: Intro

Comparative designs are used to:

  • apply existing theory to new cases
  • develop new theory or hypotheses
  • test existing theories through real-world variation

Small-N comparisons involve analyzing two or more cases in depth. They are valuable for:

  • uncovering causal mechanisms through process tracing
  • offering rich contextualization
  • studying rare or hard-to-observe outcomes

Comparative Analysis

Small-N Comparative Designs Common Strategies

  • Most Similar Systems Design:
    Cases that are similar in most relevant ways, but differ in the key variable of interest.
    → Helps isolate the effect of that one difference.
  • Most Different Systems Design:
    Cases that are different in most respects but share the same outcome.
    → Helps identify a common cause.

Comparative Analysis

Small-N Comparative Designs Examples:

  • Lipset (1959) – Social Requisites of Democracy in Europe & South America
  • Skocpol (1979) – Revolutions in Russia, France, and China
  • Lewis-Beck (1986) – Economic Voting in Britain, France, Germany, and Italy

Types of Research Designs

Experimental Designs

Most rigorous way to test if X causes Y. Why?

  • You control who gets treatment vs. control
  • Randomization eliminates confounders

Basic Steps

  1. Split into treatment and control groups
  2. Randomly assign participants
  3. Measure outcome before the intervention (pre-test)
  4. Apply the treatment
  5. Measure outcome again (post-test)

Types of Research Designs

Natural Experiments

Assignment to treatment is determined by some external, plausibly exogenous factor — not by the researcher.

Key designs:

  • differences-in-difference (based on the parallel trends assumption)
  • regression-discontinuity design (based on the continuity assumption)

These methods try to mimic experiments, even without full control.

Types of Research Designs

Natural Experiments: Assumptions

Quasi-experiments need:

  • Variation in the causal variable that’s independent of confounders
  • That variation must be as good as random

Hard to guarantee in practice — strong designs are needed.

Example:

  • policy reforms affecting only some regions
  • eligibility thresholds

Note: These designs often have high internal validity, but limited generalizability (lower external validity)

Methods

Using the Model Article: Data Description Homework 1

Let us examine the model article that we identified

Use some of the language that the authors are using, but don’t write more than 3-4 sentences per question.

  • What data are the authors using?
  • Where is it from?
  • How novel is it?

Methods

Using the Model Article: Data Description

Murray, 2014.
What data are the authors using?

The article is primarily a normative and theoretical piece. It draws on existing literature, theoretical arguments, and some secondary empirical evidence, such as studies on candidate backgrounds, legislative performance, and gender balance effects on policy and institutions.

Where is it from?

The data and examples are pulled from prior studies across Western democracies, particularly the UK, as well as secondary analyses from countries like Mexico, France, and Sweden. The references include works by scholars like Dahlerup, Lovenduski, Norris, and Franceschet.

How novel is it?

The argument is highly novel in its framing: instead of promoting “quotas for women,” the author proposes explicit “quotas for men” to combat overrepresentation. This reframing aims to shift the burden of justification from women to men and challenge the myth of meritocracy in male-dominated politics.

Methods

Using the Model Article: Analytical Strategy Homework 2

Use some of the language that the authors are using, but don’t write more 3-4 sentences.

  • How are the authors analyzing the data?
  • How do the results help answer the question?
  • Any limitations?
  • What variables are analyzed?
  • Why were they chosen?

Methods

Using the Model Article: Analytical Strategy Example

Murray, 2014.

How are the authors analyzing the data?

The author combines normative political theory with analysis of practical implications, drawing on existing empirical studies to illustrate how male overrepresentation undermines meritocracy and quality representation. The approach is conceptual rather than based on original quantitative analysis.

How do the results help answer the question?

The analysis reframes gender quotas as a corrective to overrepresentation, arguing that reducing the number of men would expand the talent pool and improve the quality of democratic representation. This helps answer the core question of how to enhance representation for all citizens.

Any limitations?

The article lacks original empirical data and acknowledges the difficulty of measuring “merit” objectively. It calls for further research into selection criteria and representation quality, recognizing that redefining these concepts is a long-term and contested process.

Methods

Using the Model Article: Analytical Strategy

Murray, 2014.

What variables are analyzed?

Variables include gender composition of legislatures, candidate backgrounds (education, profession, prior experience), and perceived competence or merit. The article also considers symbolic and substantive representation as outcomes.

Why were they chosen?

These variables are central to debates on representation and meritocracy. They help illustrate how current systems favor a narrow subset of men and how quotas could diversify and improve the pool of elected officials.

Methods

Data Description: Homework 3

Now draft your own methods section in which you also answer:

  • What data are you using?
  • Where is it from?
  • How novel is it?

Methods

Analytical Strategy: Homework 4

Get your draft and type under the methods section the answer to the following questions:

  • How are you analyzing the data?
  • How do the results help answer the question?
  • Any limitations?
  • What variables are analyzed?
  • Why were they chosen?

Key Takeaways

  • Research design = the framework of your study (case study, experiment, etc.)
  • Methods = the tools you use to collect and analyze data
  • Good designs aim for both internal and external validity
  • Choose your method based on your question and data availability
  • Case studies and small-N comparisons help explain mechanisms
  • Experiments and natural experiments help establish causality
  • Always explain:
    • What data you’re using
    • Where it came from
    • How you’re analyzing it
    • Why it helps answer your research question