Research Design & Methodology

INTRODUCTION

Background

What do we most value in our social and political lives?  What do we think is working well, and what are our concerns?  How do we view pressing issues such as governance and taxes, health care, education, jobs and the environment?

Pundits and politicians may think they know what people on the street have to say about these and other issues, but every once in a while it’s a good idea to check in directly and find out.  With a strong public policy orientation, this project represents the third installment of the decennial Oregon Values and Beliefs Project, continuing and extending similar opinion research conducted in 1992 and 2002.  The study reaches across a broad and representative swath of Oregon’s adult population—making a point to include large groups that are typically not heard in other public forums or civic participation methodologies—to learn what residents think about important issues affecting our social and political lives.  Findings show that Oregonians may not be as divided on many issues as is commonly perceived.  There are differences, to be sure.  But the survey finds surprising areas of consensus, even when examining the much-cited urban and rural divide within Oregon.

This installment of the Oregon Values and Beliefs Project serves multiple purposes. One is to provide the public a snapshot of the beliefs held by regular Oregonians at this point in time and, through comparison to previous studies, to examine how our views on certain topics may be changing.  Another purpose is to provide information to policymakers and interest groups about how regular Oregonians feel about the most fundamental issues of the day.  The political discourse of our state, and our nation, is often driven by assumptions about public attitudes.  This project tests the validity of some of these assumptions.

Still another purpose—perhaps most important—of the project is to give voice to the large group of Oregonians who are often not asked their opinions and have no common venue in which to share their views.  Randomly contacting all citizens (not just frequent voters) through the telephone and internet allows us to fulfill this aim.

Methodology

The Oregon Values and Beliefs Project was led by DHM Research, a Portland firm with more than three decades of experience in communications, marketing, and surveying for clients in the Pacific Northwest and throughout the country.  PolicyInteractive Research, a Eugene nonprofit organization that focuses on values-based opinion research, also participated in conducting the project.  Both firms are independent and non-partisan.  Sponsors include Oregon Health & Science University, The Oregon Community Foundation, Oregon State University, and Oregon Public Broadcasting.

The study has two basic tracks. Track 1 consists of three statewide surveys conducted in April and May 2013.  Collectively the surveys asked 198 questions of randomly selected Oregonians ages 18 and older.  Final sample sizes were 3,971 respondents for Survey #1, 1,958 for Survey #2, and 1,865 for Survey #3.  Track 1 employed telephone (landline and cell) and online interviewing to mitigate the disadvantages of any single collection medium.  Enough interviews were completed in five geographic regions (Central, Eastern, Portland Metro, Southern, and Willamette Valley) to permit statistically reliable analysis at the regional level.  The research design used quotas and statistical weighting based on the U.S. Census to ensure representativeness within regions by age, gender, and income.  The regions were then weighted proportionally by population per the U.S. Census to yield statewide results.

Track 2 of the study was a public involvement process that invited all citizens to become part of Oregon’s Kitchen Table and respond to the surveys through that venue.[1]  Residents also had the option to take a shorter survey that asked a mix of questions from the three longer questionnaires.  The short-version questionnaire featured more open-ended questions, allowing citizens to offer their own views in ways not often available in longer surveys.  Researchers will use this open response data for guidance in future survey research in an effort to allow every idea to be seen and heard.  Completed surveys from the public involvement track will be merged into a master file with data collection methodologies clearly identified.  The master file will be available for public access and continued research.

Altogether, over 9,500 Oregonians participated in the 2013 Oregon Values and Beliefs Study.

 Question Design

This type of research has well known limits.  The research team acknowledges and emphasizes the impossibility of capturing the full spectrum of cultural values in any survey.  Furthermore, there will always be competing ideas about the best way to ask questions to get reliable insights.  The breadth of values covered in this project—despite nearly 200 questions—demands a broad-brush approach.  The goal is not to offer an ultimate statement of Oregon values but rather a baseline of information from which to draw inferences and a platform for more detail as circumstances permit.

The surveys employed closed and open-ended question formats.  Closed questions require the respondent to choose from a set of pre-selected answers that generally represent the spectrum of a response scale, e.g., from “very important” to “very unimportant.”  Open questions ask respondents to volunteer answers using their own words, a kind of “top-of-the-mind” response.  Both question formats have advantages and disadvantages.  Closed questions allow for more accurate statistical analysis, but involve pre-packaged responses on commonly recognized topics.  Open questions allow respondents to express their own views (within some space limits), but are more costly to administer and involve more subjectivity in analysis.  The Track 1 surveys typically placed open questions at the beginning, using them to focus on broad, subjective views about Oregon, leadership opportunities, and general concerns.  Open and closed question formats complement each other, with the more subjective responses deepening our understanding of fixed categories while the more objective responses provide a check or validation of subjective views.

The Oregon Values and Beliefs Study stands out from others in that, when asking questions about policy priorities with cost implications, it informed respondents that rating an item as “important” or “desirable” implied a willingness to support some increase in taxes or reallocation of tax dollars from other public services.  Paper and online versions of the surveys used $ symbols to emphasize the real-world implications of policy preferences.

The OVB project is also distinctive in that key topics recur across the three surveys in a variety of question formats and contexts.  This “test-retest” or “triangulation” method improves our confidence in the validity of responses.

The research team designed the surveys to be impartial and nonpartisan.  Many of the questions focus pointedly on difficult issues.  Where we anticipated cultural disagreement, we paid considerable attention to provide balance in tone and weight.

Question design went through multiple iterations over months of development.  Pre-testing questions for clarity, balance, bias and potential for error involved a range of editorial expertise and assessment.  A pilot non-representative sample of 450 respondents was employed for the public involvement track.

Study Materials

The 2013 study includes many benchmark questions from the two previous studies, which allow for instructive comparisons.  And this year we also delved more deeply into specific issues.  The survey gauges the values and beliefs of Oregonians in thirteen basic categories:

  • Charitable giving
  • Economy and the environment
  • Economy and jobs
  • Education
  • Energy
  • Environment
  • Health care
  • Land use
  • Low-income support services
  • Public safety and law enforcement
  • Religion
  • Taxation
  • Transportation

The research team has prepared summaries of key findings on each of these topics. The key findings and conclusions found in these documents reflect the judgment of the research partners and not necessarily the views of the sponsoring organizations.  We hope all Oregonians find this survey helpful as we work collectively to shape our future.

Overall Results

Click on the links below to access a PDF of the survey indicated.

Survey #1
Survey #2
Survey #3


[1] For information about Oregon’s Kitchen Table, visit www.oregonskitchentable.org   We encourage Oregonians to become part of this independent, non-partisan effort to help shape the state’s future.