Skip to content
Insights Engineer

← All field guides

Field guide № 001 · An Insights Engineer field guide

A Century of Asking

Market research from Parlin’s department at Curtis Publishing in 1911 to the language model. Twelve decades, the instrument that defined each one, and the people who brought it.

12 eras, 53 dated events, 42 sourcesLast checked 27 October 2026

100%
Instrument

Hover a mark to read it, click to pin it. Zoom with + and −, or ⌘/Ctrl and scroll. Pan by dragging or with the arrow keys; the timeline zooms along the years.

Twelve decades on one axis. The band at the top is the decade, the dotted lines are the five kinds of instrument, and each mark is a dated event with a source at the foot of the page. Pick a mark to read it here.

Market research is a young discipline with an old habit. The habit is the study: a question, a method, a wait, a report. It formed when answers were expensive, and every instrument of the last century, from the door-to-door interview to the online panel, made the study cheaper without changing its shape.

Read the plates below as one curve. Each instrument makes asking cheaper, the arguments repeat, and the function never quite asks what it is for. Until asking becomes nearly free and the question becomes unavoidable.

The plates

  1. Plate 01

    1910s

    1911 to 1919

    A publisher hires a schoolteacher to find out who reads the adverts

    Instrument · Survey & sample
    The field countGo to the towns, walk the stores, count what is on the shelves and who buys it, type it up.
    Artefact
    Department Store Lines, 1912Parlin's report on what department stores sold: several hundred typed pages, compiled by visiting stores in dozens of cities and listening to the buyers.
    Who
    • Charles Coolidge Parlin, manager, Division of Commercial Research, Curtis Publishing, from 1911
    • Cyrus H. K. Curtis, publisher who paid for the department so his magazines could sell advertising with numbers
    • J. Walter Thompson Company, the agency whose research staff published Population and Its Distribution in 1912
    Dated events
    • 1911Curtis Publishing hires Parlin: the first commercial research department2
    • 1911Parlin's first study, on the agricultural implement trade2
    • 1912J. Walter Thompson publishes Population and Its Distribution39
    • 1912Department Store Lines: Parlin's report on what the stores sold2
    • 1919C. S. Duncan's Commercial Research, the first textbook of the trade1
    Carried forward
    The idea that a business could know its market by going and counting it.

    Dates from Lockley (1950) and Ward (2010). Parlin's early reports are described in both; the JWT census volume is in the Hartman Center at Duke. Sources 1, 2, 39.

  2. Plate 02

    1920s

    1920 to 1929

    The door-to-door interview, the recognition test and the first measurement company

    Instrument · Interview
    The door-to-door interviewA trained interviewer, a printed list of questions, and a doorstep. Answers pencilled in and tallied back at head office.
    Artefact
    A P&G field questionnaireA few printed questions about soap, asked on the doorstep by a woman in a hat, answered in pencil, tallied in Cincinnati.
    Who
    • Paul 'Doc' Smelser, economist who built Procter & Gamble's market research department from 1925
    • Daniel Starch, psychologist whose recognition test gave an advertisement a score
    • Arthur C. Nielsen, engineer who incorporated the A. C. Nielsen Company in Chicago in 1923
    Dated events
    • 1923Arthur C. Nielsen incorporates the A. C. Nielsen Company in Chicago5
    • 1923Starch's Principles of Advertising sets out the recognition test6
    • 1924Procter & Gamble begins consumer research; Smelser's department follows in 19254
    • 1928George Gallup's doctorate on measuring what newspaper readers actually read3
    Carried forward
    Research as a department inside the brand, not a favour from the publisher. Measurement as a business.

    P&G's start in consumer research follows Rising Tide (2004). Starch's method is set out in his own 1923 textbook. Nielsen's founding year is the company's own. Sources 4, 5, 6, 3.

  3. Plate 03

    1930s

    1930 to 1939

    Sampling beats a mailing list of millions, and the radio dial gets a meter

    Instrument · Survey & sample
    The drawn sampleA few thousand people chosen on purpose beat two million who chose themselves. The decade's other instrument, the Audimeter, recorded instead of asking.
    Artefact
    The Audimeter tapeA strip inside a box bolted to the family radio, scratched with a line every time the dial moved, posted back to Chicago each month.
    Who
    • George Gallup, founded the American Institute of Public Opinion in 1935; called the 1936 election right
    • Archibald Crossley, began telephone radio ratings in 1930 and gave the industry the word 'rating'
    • Arthur C. Nielsen, bought the Audimeter in 1936 and built a retail audit around what left the shelf
    Dated events
    • 1930Crossley's Cooperative Analysis of Broadcasting: the first radio ratings, by telephone7
    • 1933The Nielsen Drug Index audits what leaves the shelf; the Food Index follows in 19345
    • 1935Gallup founds the American Institute of Public Opinion8
    • 1936The Literary Digest mails millions of postcards and calls the election wrong; Gallup calls it right3
    • 1936Nielsen acquires the Audimeter5
    • 1938Hooper's telephone coincidental: what are you listening to right now?7
    Carried forward
    Who you ask matters more than how many. And the behavioural record is better than the recollection.

    The ratings history follows Beville (1988). Gallup's own account is in The Pulse of Democracy (1940) and Converse (1987) covers the Literary Digest. Nielsen dates are the company's. Sources 3, 5, 7, 8.

  4. Plate 04

    1940s

    1940 to 1949

    A sociologist drags a colleague out of a dinner party, and the focus group is born

    Instrument · Interview
    The focused interviewPlay the programme, record the reaction, then ask about the exact moment the reaction happened. The discussion guide descends from this.
    Artefact
    The Lazarsfeld-Stanton Program AnalyzerA box with a green button for like and a red one for dislike, pressed by listeners while a programme played. The tape was read back to them in the interview afterwards.
    Who
    • Paul Lazarsfeld, ran Columbia's Office of Radio Research; co-invented the Program Analyzer with Frank Stanton
    • Robert K. Merton, sociologist who wrote the focused interview down and later regretted the focus group
    • Herta Herzog, asked what listeners got from daytime serials, not what the serials did to them
    • Ernest Dichter, founded the Institute for Motivational Research in 1946 around the depth interview
    Dated events
    • 1941Lazarsfeld brings Merton to a Program Analyzer session; the focused interview begins9
    • 1942The Nielsen Radio Index goes commercial5
    • 1944Herzog publishes on what daytime serial listeners get from the serials11
    • 1946Merton and Kendall publish 'The Focused Interview'10
    • 1946Dichter founds the Institute for Motivational Research12
    Carried forward
    The depth conversation as a research instrument. The idea that the audience has purposes of its own.

    The dinner-party story is Merton's own, told in Public Opinion Quarterly in 1987. The Program Analyzer and Herzog's study are in Radio Research 1942 to 1943. Dichter's institute is dated by Schwarzkopf and Gries (2010). Sources 9, 10, 11, 12, 5.

  5. Plate 05

    1950s

    1950 to 1959

    Motivation research goes mainstream, gets rich, and gets a backlash

    Instrument · Interview
    The depth interviewTwo hours on a couch about a bar of soap. Sold by the study, bought by every packaged-goods company with a budget.
    Artefact
    The paperback of The Hidden PersuadersA million copies sold. The first time the people being studied read what the studiers were up to.
    Who
    • Ernest Dichter, the most famous researcher in America by mid-decade
    • Alfred Politz, physicist turned researcher who attacked Dichter in public and ran Life's readership studies
    • Wendell R. Smith, made the case for market segmentation in the Journal of Marketing, 1956
    • Vance Packard, journalist whose The Hidden Persuaders told the public what the researchers were doing
    Dated events
    • 1950The Nielsen Television Index begins5
    • 1953Politz's A Study of Four Media for Life magazine15
    • 1956Wendell Smith: product differentiation versus market segmentation14
    • 1957The Hidden Persuaders is published13
    Carried forward
    Segmentation. The rigour-versus-depth argument. And the first warning that the public is watching the watchers.

    Dichter and the backlash follow Schwarzkopf and Gries (2010) and Packard's own book. Politz's media study was published by Time Inc. in 1953. Smith's article is the origin of segmentation as a term of trade. Sources 12, 13, 14, 15, 5.

  6. Plate 06

    1960s

    1960 to 1969

    Theory arrives: myopia, trade-offs, and the railroad business

    Instrument · Theory & metric
    The trade-offStop asking people to rate things. Make them choose between things and read what they give up.
    Artefact
    Journal of Marketing Research, vol. 1, no. 1February 1964. The trade gets a journal of its own, and a place to argue about method in print.
    Who
    • Theodore Levitt, harvard Business School; wrote Marketing Myopia in 1960
    • David Ogilvy, former Gallup interviewer who made research respectable in the creative department
    • R. Duncan Luce and John W. Tukey, mathematicians whose 1964 paper became conjoint analysis
    Dated events
    • 1960Levitt's Marketing Myopia: the railroads thought they were in the railroad business16
    • 1962Rogers publishes Diffusion of Innovations41
    • 1963Ogilvy's Confessions of an Advertising Man17
    • 1964Luce and Tukey publish simultaneous conjoint measurement18
    • 1964The Journal of Marketing Research publishes its first issue40
    Carried forward
    Define the work by the need. Learn value from trade-offs, not ratings.

    The three papers and two books are primary sources and are listed below. The journal's first issue is dated from the AMA's own archive. Sources 16, 17, 18, 40, 41.

  7. Plate 07

    1970s

    1970 to 1979

    The computer picks up the telephone, and the checkout starts keeping score

    Instrument · Survey & sample
    The computer-assisted telephone interviewThe questionnaire on a screen, the interviewer reading it, the answers keyed straight in. Within fifteen years it was how most survey data was collected.
    Artefact
    A packet of Wrigley's Juicy FruitScanned at Marsh Supermarket in Troy, Ohio, on the morning of 26 June 1974. The first retail barcode transaction. Almost nobody noticed.
    Who
    • Chilton Research Services, ran the first CATI survey, for AT&T, in 1971
    • Paul E. Green and Vithala R. Rao, brought conjoint measurement into marketing research in 1971
    • Andrew Ehrenberg, showed in Repeat-Buying (1972) that brand buying follows regular laws
    Dated events
    • 1971Chilton runs the first computer-assisted telephone interview, for AT&T22
    • 1971Green and Rao bring conjoint measurement into marketing research19
    • 1972Ehrenberg's Repeat-Buying20
    • 1974The first retail barcode scan, Troy, Ohio21
    • 1979Information Resources Inc. is founded; BehaviorScan test markets follow23
    Carried forward
    Computer-assisted fieldwork. Behavioural data as a by-product of commerce. The empirical laws of buying.

    The CATI date is from Nicholls and Groves (1986). The first scan is GS1's own account. Conjoint and Repeat-Buying are primary sources. IRI's founding and BehaviorScan are described in the Lodish meta-analysis. Sources 19, 20, 21, 22, 23.

  8. Plate 08

    1980s

    1980 to 1989

    Scanner panels, people meters and the industrialisation of the tracker

    Instrument · Meter & panel
    The continuous panelHouseholds that scan what they buy and press a button when they watch. A weekly record, sold on subscription, read in a quarterly deck.
    Artefact
    The people meter handset, 1987One button per member of the household, pressed when you sat down to watch. Television measurement stops being a box and becomes a person.
    Who
    • Gerald Eskin and John Malec, founders of Information Resources Inc., whose BehaviorScan made test marketing measurable
    • Nielsen, introduced the people meter in 1987: a button per viewer instead of a box per household
    • Millward Brown, maurice Millward and Gordon Brown's firm, whose continuous tracker became the model for the brand tracking business
    Dated events
    • 1980IRI's BehaviorScan: scanner panels and split-cable television in matched test towns23
    • 1984SPSS/PC+ puts statistical software on the researcher's own desk24
    • 1987Nielsen introduces the people meter5
    Carried forward
    Continuous behavioural panels. The tracker. The study as a quarterly ritual.

    BehaviorScan is dated from the Lodish meta-analysis of its experiments. The people meter date is Nielsen's. SPSS/PC+ and Millward Brown are dated from the companies' own histories. Sources 23, 24, 25, 5.

  9. Plate 09

    1990s

    1990 to 1999

    The survey goes online and the sample goes to anyone who will click

    Instrument · Survey & sample
    The web surveyA questionnaire on a web page, sent to a panel of people who signed up for points. Fieldwork in days, then hours.
    Artefact
    A SurveyMonkey questionnaire, 1999Ten questions, built in an afternoon by someone who had never run a survey, sent to a mailing list before anyone checked the wording.
    Who
    • Greenfield Online, founded in 1994, one of the first online research panels
    • Gerald Zaltman, harvard Business School; patented the metaphor elicitation technique in 1995
    • Frederick Reichheld, bain; The Loyalty Effect (1996) set up the single-question metric
    • Ryan Finley, founded SurveyMonkey in 1999
    Dated events
    • 1994Greenfield Online is founded: the online panel begins35
    • 1995Zaltman patents the metaphor elicitation technique26
    • 1996Reichheld's The Loyalty Effect27
    • 1999SurveyMonkey: the self-serve survey34
    Carried forward
    Fast, cheap, self-serve fieldwork. The panel as a commodity. The first hint that quantity would not be the problem.

    Greenfield Online's founding is from its filings. The ZMET patent is a primary source. SurveyMonkey's founding year is the company's own. Sources 26, 27, 34, 35.

  10. Plate 10

    2000s

    2000 to 2009

    One question, a Nobel, and the do-it-yourself decade

    Instrument · Theory & metric
    The single metricOne question, a subtraction, a number the board can repeat. Adopted faster than the researchers could object.
    Artefact
    The 0-to-10 scale on a receipt'How likely are you to recommend us to a friend?' Printed at the bottom of the bill, asked after every call, averaged into a number the board could repeat.
    Who
    • Frederick Reichheld, published the Net Promoter Score in the Harvard Business Review in 2003
    • Daniel Kahneman, nobel in 2002; gave the field a vocabulary for why saying and doing diverge
    • Qualtrics, founded in Provo, Utah in 2002; made every department its own research function
    Dated events
    • 2002Qualtrics is founded34
    • 2002Kahneman's Nobel brings behavioural economics into marketing29
    • 2003Reichheld publishes the Net Promoter Score28
    • 2008Thaler and Sunstein's Nudge30
    Carried forward
    The single metric. Behavioural economics. Research as something anyone in the company could do, and did.

    NPS and Nudge are primary sources. The Nobel citation is the foundation's. Qualtrics's founding is the company's own. Sources 28, 29, 30, 34.

  11. Plate 11

    2010s

    2010 to 2019

    How Brands Grow, the automated platform, and analytics overtakes research

    Instrument · Software & model
    The automated platformThe agency's methods, templated and sold on subscription. A concept test in a day, a conjoint in a week, a deck nobody reads twice.
    Artefact
    The research repositoryA searchable folder of every study the team has ever run. Most were opened once, on the day they were presented.
    Who
    • Byron Sharp, ehrenberg-Bass Institute; How Brands Grow (2010) put Ehrenberg's laws in plain English
    • Steve Phillips, founded ZappiStore in 2012, agency methods on subscription
    • Kate Towsey, founded the ResearchOps Community in 2018
    Dated events
    • 2010Sharp's How Brands Grow31
    • 2012ZappiStore: agency methods sold on subscription36
    • 2018The ResearchOps Community is founded37
    • 2019ESOMAR widens its count of the industry; analytics is now the larger half32
    Carried forward
    Automation of the method. Analytics as the bigger sibling. The repository. A budget still counted in studies.

    How Brands Grow is a primary source. Zappi and ResearchOps dates are the organisations' own. Industry size and the analytics share are from ESOMAR's Global Market Research reports. Sources 31, 32, 36, 37.

  12. Plate 12

    2020s

    2020 to 2026

    The pandemic takes fieldwork online for good, and the language model takes the middle of the process

    Instrument · Software & model
    The language modelTranscribes, codes, summarises and drafts the deck in an afternoon. Then moderates the interview and, if you let it, answers it too.
    Artefact
    A transcript, coded before lunchForty minutes of interview, transcribed, themed and drafted into a deck by a model, with the researcher's afternoon left over for the question.
    Who
    • OpenAI, released ChatGPT on 30 November 2022; the middle of the research process moved within a year
    • The insights teams in GreenBook's GRIT panels, who went from a minority using generative AI to a large majority inside two years
    • The insights engineer, the role that builds the system that answers the question. It did not have a name until this site gave it one
    Dated events
    • 2020The pandemic moves what was left of in-person fieldwork online, in a quarter33
    • 2022ChatGPT is released38
    • 2024AI-moderated interviews and synthetic respondents become ordinary tools33
    • 2026The role gets a name: the insights engineer42
    Carried forward
    The end of the study as the product. The beginning of the role.

    The move online and the adoption curve are GreenBook's GRIT figures. ChatGPT's release date is OpenAI's. The last event is this site's own claim. Sources 33, 38, 42.

Sources

Dates and attributions follow the organisations’ own records and the published histories below. Where sources disagree on a year, the plate says “early” or “mid” rather than choosing. Last checked 27 October 2026.

  1. 1Lawrence C. Lockley, "Notes on the History of Marketing Research", Journal of Marketing, vol. 14, April 1950.
  2. 2Douglas B. Ward, A New Brand of Business: Charles Coolidge Parlin, Curtis Publishing Company, and the Origins of Market Research (Temple University Press, 2010).
  3. 3Jean M. Converse, Survey Research in the United States: Roots and Emergence 1890 to 1960 (University of California Press, 1987).
  4. 4Davis Dyer, Frederick Dalzell and Rowena Olegario, Rising Tide: Lessons from 165 Years of Brand Building at Procter & Gamble (Harvard Business School Press, 2004).
  5. 5Nielsen, company history (nielsen.com): incorporation in 1923, the Drug and Food Indexes of 1933 and 1934, the Audimeter in 1936, the Radio Index in 1942, the Television Index in 1950, the people meter in 1987.
  6. 6Daniel Starch, Principles of Advertising (A. W. Shaw, 1923).
  7. 7Hugh Malcolm Beville Jr., Audience Ratings: Radio, Television, Cable (Lawrence Erlbaum, revised edition 1988), on Crossley's Cooperative Analysis of Broadcasting and the Hooper ratings.
  8. 8George Gallup and Saul Forbes Rae, The Pulse of Democracy (Simon & Schuster, 1940).
  9. 9Robert K. Merton, "The Focussed Interview and Focus Groups: Continuities and Discontinuities", Public Opinion Quarterly, vol. 51, 1987.
  10. 10Robert K. Merton, Marjorie Fiske and Patricia L. Kendall, The Focused Interview (Free Press, 1956); first described by Merton and Kendall in the American Journal of Sociology, 1946.
  11. 11Paul F. Lazarsfeld and Frank N. Stanton (eds), Radio Research 1942 to 1943 (Duell, Sloan and Pearce, 1944), including Peterman on the Program Analyzer and Herzog on daytime serial listeners.
  12. 12Stefan Schwarzkopf and Rainer Gries (eds), Ernest Dichter and Motivation Research (Palgrave Macmillan, 2010).
  13. 13Vance Packard, The Hidden Persuaders (David McKay, 1957).
  14. 14Wendell R. Smith, "Product Differentiation and Market Segmentation as Alternative Marketing Strategies", Journal of Marketing, vol. 21, July 1956.
  15. 15Alfred Politz Research, A Study of Four Media (Time Inc., 1953).
  16. 16Theodore Levitt, "Marketing Myopia", Harvard Business Review, July to August 1960.
  17. 17David Ogilvy, Confessions of an Advertising Man (Atheneum, 1963).
  18. 18R. Duncan Luce and John W. Tukey, "Simultaneous Conjoint Measurement: A New Type of Fundamental Measurement", Journal of Mathematical Psychology, vol. 1, 1964.
  19. 19Paul E. Green and Vithala R. Rao, "Conjoint Measurement for Quantifying Judgmental Data", Journal of Marketing Research, vol. 8, August 1971.
  20. 20A. S. C. Ehrenberg, Repeat-Buying: Theory and Applications (North-Holland, 1972).
  21. 21GS1 US, on the first retail scan of a UPC barcode at Marsh Supermarket, Troy, Ohio, 26 June 1974.
  22. 22William L. Nicholls II and Robert M. Groves, "The Status of Computer-Assisted Telephone Interviewing", Journal of Official Statistics, vol. 2, 1986, which dates the first CATI system to Chilton Research Services in 1971.
  23. 23Leonard M. Lodish and others, "How T.V. Advertising Works: A Meta-Analysis of 389 Real World Split Cable T.V. Advertising Experiments", Journal of Marketing Research, vol. 32, May 1995, on IRI's BehaviorScan.
  24. 24SPSS Inc. (now IBM SPSS), product history: SPSS/PC+ for the IBM PC, 1984.
  25. 25Millward Brown, company history: founded in Leamington Spa in 1973 by Maurice Millward and Gordon Brown; continuous brand tracking through the 1980s.
  26. 26Gerald Zaltman, US Patent 5,436,830, "Metaphor elicitation technique with physiological function monitoring", granted 1995.
  27. 27Frederick F. Reichheld, The Loyalty Effect (Harvard Business School Press, 1996).
  28. 28Frederick F. Reichheld, "The One Number You Need to Grow", Harvard Business Review, December 2003.
  29. 29The Nobel Foundation, the Sveriges Riksbank Prize in Economic Sciences 2002, awarded to Daniel Kahneman and Vernon L. Smith.
  30. 30Richard H. Thaler and Cass R. Sunstein, Nudge (Yale University Press, 2008).
  31. 31Byron Sharp, How Brands Grow (Oxford University Press, 2010).
  32. 32ESOMAR, Global Market Research reports, 2019 to 2023 editions, for the size of the industry and the share taken by data analytics.
  33. 33GreenBook, GRIT (GreenBook Research Industry Trends) reports, 2020 to 2025, on the move online and the adoption of generative AI by insights buyers and suppliers.
  34. 34SurveyMonkey (founded 1999 by Ryan Finley) and Qualtrics (founded 2002 in Provo, Utah), company records.
  35. 35Greenfield Online, company filings: founded 1994 in Wilton, Connecticut, one of the first online research panels.
  36. 36Zappi, company history: founded as ZappiStore in 2012 by Steve Phillips.
  37. 37ResearchOps Community, founded 2018 by Kate Towsey.
  38. 38OpenAI, "Introducing ChatGPT", 30 November 2022.
  39. 39J. Walter Thompson Company archives, John W. Hartman Center for Sales, Advertising and Marketing History, Duke University, on Population and Its Distribution (1912).
  40. 40American Marketing Association, Journal of Marketing Research, vol. 1, no. 1, February 1964.
  41. 41Everett M. Rogers, Diffusion of Innovations (Free Press, 1962).
  42. 42This site: the Insights Engineer manifesto and definition, 2026.

One row per dated event, not per paper or product. The 12 plates and 53 events are illustrative, chosen for what each instrument changed about the work, not ranked by importance. Corrections are welcome by reply to the newsletter. Field guide № 001 · Insights Engineer · last checked 27 October 2026.

One email a week

Who is hiring for the job, what they are building, and what the job is turning into.

Unsubscribe any time.