Extensions: scoring surveys without a QSF

What this answers. What you can do with a survey not built in Qualtrics, and what full support for another platform would take.

What you need. A question catalogue describing your survey (schema below).

What it cannot establish. Without routing and display-logic information, a catalogue gives per-question scores and a single-pass total, not a burden range or profile.

library(surveyBurden)
#> 
#>                   .-- Q2 -- Q3 --.
#>   survey -- Q1 --+               +-- Burden 0.2.0
#>                   '-- Q4 --------'
#> 
#> An R package for assessing the response-burden of web surveys
#> Docs: https://pmpk20.github.io/surveyBurden/
#> 
#> Please cite: King P (2026). surveyBurden: An R package for assessing
#> the response-burden of web surveys. R package version 0.2.0,
#> https://github.com/pmpk20/surveyBurden.
#> Use citation("surveyBurden") for BibTeX.

surveyBurden reads Qualtrics .qsf files, but many surveys are built on other platforms (LimeSurvey, SurveyEngine, REDCap, and so on). This vignette documents where the Qualtrics-specific code ends, what you can already do with a survey from another platform, and what supporting a new platform fully would take.

In short:

Status	What	How
Working now	Per-question GfS+ scores and a survey total for any survey you can describe as a list of questions	`validate_catalogue()` then `score_burden()`
Requires an adapter	A floor/ceiling burden band per path	build a paths table yourself, then `path_burden(NULL, paths = , scored = )`
Future interface work	Display-logic profiles, `burden_report()`, respondent burden, calculation certainty	these currently need a Qualtrics `qsf_raw` object

Three-layer architecture

The package has three layers:

Layer 1:  Parser          (platform-specific)
          read_qsf() / fetch_qsf() → parse_qsf()
                                  ↓
          Question catalogue  ← the intermediate representation
                                  ↓
Layer 2:  Path resolver     (partially platform-specific)
          resolve_flow() → resolve_paths()
                                  ↓
Layer 3:  Scorer + reporter (platform-agnostic)
          score_burden() → path_burden() → burden_report()

Layer 1 reads a Qualtrics survey and produces a standardised question catalogue: a tibble with one row per live question, in flow order, with the structural fields the scorer needs. This is the intermediate representation.

Layer 2 resolves the survey’s routing logic (branches, block randomisers, early exits) and display logic (conditional questions) into the structural paths the flow allows. This layer reads the QSF’s SurveyFlow, block and DisplayLogic elements directly, so it is Qualtrics-specific.

Layer 3 applies the GfS+ points to the catalogue, computes per-path burden bands and profiles, and assembles the report. Only part of it is independent of the QSF: score_burden() needs just the catalogue, and path_burden() needs just the catalogue and a paths table. The profile engine behind path_burden_profile() and burden_report() still reads block definitions and display-logic trees from the qsf_raw object.

The question catalogue

The catalogue is a tibble produced by parse_qsf(), one row per live question. Any parser that produces this schema can use score_burden() directly. The functions further down the pipeline need more than the catalogue; see What a new parser must provide.

Required columns

Column	Type	Description
`question_id`	character	Unique identifier for each question
`std_type`	character	Standardised question type (see below)

Structural columns (used by the scorer)

Column	Type	Description
`n_options`	integer	Number of response options (`NA` if unknown)
`n_rows`	integer	Number of matrix rows (`NA` for non-matrix)
`n_cols`	integer	Number of matrix columns (`NA` for non-matrix)
`text_words`	integer	Word count of the question stem
`is_dropdown`	logical	Single-choice rendered as a dropdown
`is_multiline`	logical	Open-text field with multi-line entry
`is_multi_answer`	logical	Matrix where each cell is a checkbox
`is_hidden`	logical	Hidden from the respondent
`options_numeric`	logical	All option labels are numbers
`label_text`	character	Lowercased, joined response labels
`question_text`	character	Plain-text question stem (for cue detection)
`has_display_logic`	logical	Whether the question is conditionally shown
`has_validation`	logical	Whether a response is forced
`in_loop`	logical	Whether the question is inside a loop block
`loop_max`	integer	Maximum loop iterations (`NA` if not in loop)
`flag`	character	Confidence: `"auto"`, `"inferred"`, `"manual"`, `"unknown"`

Provenance columns (not used by the scorer)

Column	Type	Description
`qualtrics_type`	character	Original platform question type
`selector`	character	Platform-specific rendering selector
`subselector`	character	Platform-specific sub-selector
`flow_order`	integer	Position in the survey flow
`block_id`	character	Block identifier
`block_name`	character	Block description

The qualtrics_type, selector, and subselector columns are carried for debugging and provenance. The scorer does not read them; it uses std_type and the boolean columns (is_dropdown, is_multiline, is_multi_answer) instead.

Standardised question types

The std_type column uses the following vocabulary:

`std_type`	Description	GfS+ scoring
`single_choice`	One answer from a list (radio, dropdown, NPS)	By option count
`multi_choice`	Multiple answers from a list (checkboxes)	By option count
`matrix`	Grid/table: rows x columns	Per row, by column count
`open_text`	Free text (single-line or essay)	Short vs essay
`slider`	Continuous input (visual analogue)	By label cues
`ranking`	Drag-and-drop or number ranking	Fixed
`constant_sum`	Allocate a total across categories	As numeric answer
`descriptive`	Instruction or transition text (no input)	By word count
`timing`	Page-timing metadata (not shown)	0 points
`meta`	Browser/metadata capture (not shown)	0 points
`captcha`	Bot-detection challenge	Fixed

A new parser maps its platform’s question types into this vocabulary. The mapping need not be one-to-one: several Qualtrics types map to single_choice, for example.

Validating a hand-built catalogue

validate_catalogue() checks that a data frame has the columns the scorer needs, fills missing optional columns with safe defaults, and coerces the result to a tibble:

cat <- data.frame(
  question_id = c("Q1", "Q2", "Q3", "Q4", "Q5"),
  std_type    = c("single_choice", "multi_choice", "matrix",
                  "open_text", "descriptive"),
  n_options   = c(5L, 8L, NA, NA, NA),
  n_rows      = c(NA, NA, 4L, NA, NA),
  n_cols      = c(NA, NA, 5L, NA, NA),
  text_words  = c(8L, 12L, 6L, 10L, 25L),
  is_multiline = c(FALSE, FALSE, FALSE, TRUE, FALSE),
  stringsAsFactors = FALSE
)
validated <- validate_catalogue(cat)
names(validated)
#>  [1] "question_id"        "std_type"           "n_options"         
#>  [4] "n_rows"             "n_cols"             "text_words"        
#>  [7] "is_multiline"       "max_label_words"    "label_text"        
#> [10] "options_numeric"    "question_text"      "is_hidden"         
#> [13] "is_dropdown"        "is_multi_answer"    "has_display_logic" 
#> [16] "display_logic_refs" "has_validation"     "in_loop"           
#> [19] "loop_max"           "flag"               "qualtrics_type"    
#> [22] "selector"           "subselector"        "flow_order"        
#> [25] "block_id"           "block_name"

Worked example: scoring without a QSF

This example constructs a five-question catalogue by hand and runs it through score_burden(). No QSF file is involved.

catalogue <- validate_catalogue(data.frame(
  question_id = c("Q1", "Q2", "Q3", "Q4", "Q5"),
  std_type    = c("single_choice", "multi_choice", "matrix",
                  "open_text", "descriptive"),
  n_options   = c(5L, 8L, NA, NA, NA),
  n_rows      = c(NA, NA, 4L, NA, NA),
  n_cols      = c(NA, NA, 5L, NA, NA),
  text_words  = c(8L, 12L, 6L, 10L, 25L),
  is_multiline = c(FALSE, FALSE, FALSE, TRUE, FALSE),
  question_text = c(
    "How often do you cycle?",
    "Which of these apply?",
    "Rate each mode (1-5)",
    "Describe your typical commute",
    "The next section asks about your household"
  ),
  stringsAsFactors = FALSE
))

scored <- score_burden(catalogue)
scored[, c("question_id", "std_type", "gfs_points",
           "score_flag", "score_basis")]
#> # A tibble: 5 × 5
#>   question_id std_type      gfs_points score_flag score_basis                   
#>   <chr>       <chr>              <dbl> <chr>      <chr>                         
#> 1 Q1          single_choice          2 auto       single choice, 5 options -> G…
#> 2 Q2          multi_choice          25 auto       multi-select, 8 options -> Gf…
#> 3 Q3          matrix                 8 auto       matrix, 4 rows x GfS rating <…
#> 4 Q4          open_text              6 auto       multi-line text -> GfS first …
#> 5 Q5          descriptive            2 auto       transition/instruction text: …

The function score_burden operates entirely on the question catalogue so that it does not know or care that these questions came from a hand-built data frame rather than a Qualtrics .qsf.

Survey-level burden

If using the validate_catalogue approach to calculate the burden of a non-Qualtrics survey, you could sum the per-question scores for the survey-level total:

total_points <- sum(scored$gfs_points, na.rm = TRUE)
total_minutes <- total_points / gfs_scheme()$points_per_minute
cat(sprintf("Total: %.0f GfS+ points, ~%.0f minutes\n",
            total_points, total_minutes))
#> Total: 43 GfS+ points, ~4 minutes

This is the burden of one path that shows every question once. For a survey with conditional questions it is an upper bound on a single pass – the burden a respondent would face if every question were shown – but not if questions repeat in a loop: multiply looped questions by their iteration count yourself.

What this gives you and what it does not

A hand-built catalogue scored with score_burden() gives you:

Per-question GfS+ burden points, using the same scoring rules as burden_report().
A survey-level total (the sum() above).
A per-question confidence flag (score_flag) saying how the score was reached.

It does not give you:

A burden range across branching paths – that requires a paths table (see the adapter below).
A display-logic profile – the structural min/median/max within a path. That currently requires a Qualtrics survey.
Respondent burden – respondent_burden() currently requires a Qualtrics survey as well as response data.

The total from a hand-built catalogue is a defensible measure of survey burden for any survey that can be expressed as a list of questions with their types and option counts. It applies the same published GfS+ point scheme (Heimgartner and Axhausen, 2024) as the full Qualtrics pipeline.

What a new parser must provide

Working now: a question catalogue

A data frame matching the schema above, passed through validate_catalogue(). This is all score_burden() needs: per-question scores and a survey total work immediately, as in the worked example.

Requires an adapter: a paths table

path_burden() gives a floor/ceiling band per path without a QSF if you pass it the scored catalogue and a paths table with one row per path: path_id, terminates_early, block_ids (list), q_always (list of question ids always shown on that path), q_maybe (list of question ids that may be shown) and n_gates.

paths <- tibble::tibble(
  path_id          = 1:2,
  terminates_early = c(FALSE, FALSE),
  block_ids        = list("B1", c("B1", "B2")),
  q_always         = list(c("Q1", "Q2", "Q3"), c("Q1", "Q2", "Q3", "Q4")),
  q_maybe          = list(character(0), "Q5"),
  n_gates          = c(0L, 1L)
)
path_burden(NULL, paths = paths, scored = scored)[,
  c("path_id", "gfs_floor", "gfs_ceiling")]
#> # A tibble: 2 × 3
#>   path_id gfs_floor gfs_ceiling
#>     <int>     <dbl>       <dbl>
#> 1       1        35          35
#> 2       2        41          43

Building that table is the adapter’s job: enumerating the platform’s paths and deciding, for each, which questions are always or maybe shown. Loop multiplication in the ceiling uses the catalogue’s loop_max.

Future interface work: routing and display logic

Everything beyond the band – path_burden_profile(), burden_report(), respondent_burden() and calculation_certainty() – currently takes a qsf_raw object and reads its SurveyFlow, block and DisplayLogic elements directly. Supporting another platform there needs one of:

Translation: convert the platform’s survey into a qsf_raw-shaped object – flow nodes (Standard, Branch, EndSurvey, BlockRandomizer, Group, EmbeddedData), block definitions, and question payloads with Qualtrics-format DisplayLogic trees. This works with the current code but is a substantial mapping.
A platform-neutral interface (not yet built): letting these functions accept a catalogue, a paths table and parsed display-logic predicates in place of qsf_raw.

Setting has_display_logic = FALSE for every question does not skip the display-logic profile: it tells the package the question is always shown, so conditional questions count on every path and burden is overstated. It removes conditionality from the model rather than modelling it.

Where platform-specific code lives

Function	Layer	Platform-specific?
`read_qsf()` / `fetch_qsf()`	1	Yes – reads QSF format
`classify_question()`	1	Yes – maps Qualtrics type codes
`parse_qsf()`	1	Yes – orchestrates QSF parsing
`resolve_live_blocks()`	1-2	Yes – reads QSF block definitions
`resolve_flow()`	2	Yes – walks QSF `SurveyFlow` nodes
`parse_display_logic()`	2	Yes – reads QSF `DisplayLogic` trees
`validate_catalogue()`	boundary	No – validates the intermediate representation
`score_burden()`	3	No – operates on the catalogue
`gfs_scheme()`	3	No – scoring parameters
`classify_reachability()`	2-3	Partly – catalogue fields, but ruling questions out needs Qualtrics-format `DisplayLogic` trees
`path_burden()`	3	No, given a paths table and the scored catalogue
`path_burden_profile()`	3	Yes, at present – its engine reads blocks and display logic from `qsf_raw`
`respondent_burden()`, `calculation_certainty()`	3	Yes, at present – take `qsf_raw`
`burden_report()`	3	Orchestrator – takes `qsf_raw` and threads it through Layers 1-2

The boundary between platform-specific and platform-agnostic code currently sits at the question catalogue for scoring, and at a paths table for the floor/ceiling band. Everything else still reads the QSF.

Design rationale

surveyBurden is a Qualtrics-first tool and does not claim to be platform-agnostic. The scoring rules are already separate from the parser, so scoring another platform’s survey needs only a catalogue. Full support would additionally need:

A parser that produces the question catalogue (the main work).
Routing: either translation into Qualtrics flow nodes, or the platform-neutral interface above.
Display logic, likewise, if the platform has per-question visibility conditions.

For a platform with no branching, loops or conditional questions, the catalogue and score_burden() already give the full answer.

vignette("gfs-scoring") – the scoring rules the catalogue feeds.
?validate_catalogue – the catalogue checks and defaults.