Skip to contents

Changelog

Source: NEWS.md

surveyBurden (development version)

  • burden_report(), path_burden_profile() and calculation_certainty() accept max_condition_states, the joint display-condition enumeration limit per component. The default rises from 5,000 to 10,000 states; raising it can change burden profiles and certainty counts. Certainty output records the chosen limit.

  • Breaking: one respondent-level measure. respondent_burden(qsf, responses) now returns respondent burden: the GfS+ points of every item displayed to each respondent, reconstructed from their answers, with descriptive text and questions left blank included and each loop iteration counted. by = "block" gives one row per respondent per block, the input for exposure_person_period().

    • realised_burden() (answer-based points) and realised_exposure() are removed. realised_exposure(qsf, x) is now respondent_burden(qsf, x, by = "block").
    • The previous respondent_burden(qsf, routes) (median display-logic burden of each respondent’s matched path) and its per-path table are removed, with summary_line.respondent_burden().
    • Block-level columns drop the shown_ prefix: points, items, response_items, pages, hidden_items.
    • burden_report() takes responses instead of routes, and reports $respondents (respondent burden over complete responses) instead of $population.
    • validate_times() loses basis: it always uses respondent burden and needs response columns that map to the survey’s questions.
    • words_per_line carries over: NULL, one value, or one value per respondent (for example by device).
    • The vignette realised-burden is now respondent-burden.

  • validate_times() gains basis = "shown": each respondent’s burden is the GfS+ points of every item displayed to them (summed shown_points from realised_exposure()), i.e. the respondent burden on their route, including descriptive text and questions left blank. Recommended when the export allows the flow to be replayed; "auto" is unchanged.

  • realised_burden() and realised_exposure() now map response columns through the ImportId row of a raw Qualtrics CSV export, so columns with custom export tags (e.g. w4_q19x1) are found. Display-order, page-timing and browser-meta columns no longer count as answers, and n_unmapped_cols counts only question columns that could not be mapped.

  • realised_burden() and realised_exposure() accept the path to a CSV export as well as a data frame.

  • validate_times() gains basis: by default it compares each respondent’s realised burden (what they answered) with their time, and falls back to the route prediction only when the columns do not map. It keeps finishers only (finished_only = TRUE), returns cor = NA without a warning when every prediction is the same, and prints a short summary.

  • A BlockRandomizer that only assigns embedded data (random condition assignment) no longer warns, and a fire-and-forget WebService flow step is skipped silently; one that sets fields is noted.

  • The route-mismatch warning from respondent_burden() no longer shows NA for its fallback counts.

  • respondent_burden() matches routes only on optional blocks that have a visit_ column; others (e.g. text-only blocks, whose visits leave no answer) are unknown rather than “not visited”. This fixes realised_burden()’s predicted_points, which on a survey with text-only optional blocks fell back to the plainest path for nearly every respondent.

  • respondent_burden() no longer fails on a Loop & Merge block that loops over a fixed list (no driving question).

  • QID-named timing and meta columns (QID5_FIRST_CLICK, as qualtRics::fetch_survey(import_id = TRUE) names them) are no longer counted as answers. The docs now recommend the raw CSV export, or fetch_survey(import_id = TRUE).

  • survey_summary(), burden_report()$survey and both printouts now report the number of pages (n_pages): each live block starts a page and each page break adds one; empty pages are not counted.

  • The report headline is split into three lines: “Median burden” (points and range), “Expected minutes” and “Relative to benchmark”. The burden table’s ~Min column is now “Expected Minutes”.

  • Breaking: “instrument” is now “survey” throughout. instrument_summary() is renamed survey_summary(), and the burden_report() element $instrument is now $survey (printed under “Survey”).

  • library(surveyBurden) now prints a short banner with the version, the package title and how to cite it (silence with suppressPackageStartupMessages()).

  • citation("surveyBurden") now takes its title and version from DESCRIPTION.

  • Display logic now applies And before Or among the conditions inside one group, as Qualtrics documents: A Or B And C is read as A | (B & C), not left to right as (A | B) & C. This affects path_burden_profile(), the “never shown” check and realised_exposure(). Path profiles for the bundled demo surveys are unchanged.

  • resolve_flow() now detects consecutive branches that test the same embedded-data field for equality and treats them as mutually exclusive (one-of-k) instead of independent binary decisions. For surveys that route respondents by an assignment field (e.g. stated-choice module selection), this can reduce the path count from 2^k to k+1 per group.

  • resolve_flow() now evaluates consecutive branches on the same embedded-data field jointly for any single comparison (=, !=, >, >=, <, <=), enumerating only the combinations of outcomes some value of the field can produce. Complementary gates such as > 0 and = 0 can no longer both be taken, which removes impossible paths (e.g. a respondent shown both the owner and the non-owner blocks).

  • Field values stay unrestricted, so the enumeration is an upper bound: a “neither” outcome (e.g. a negative or empty field under > 0 / = 0) is kept. resolve_flow() now reports branch fields that question JavaScript assigns with setEmbeddedData(), naming the field and question, since that code may limit the field’s values and make some enumerated paths impossible in practice. The check is textual and informational only.

  • summary_line() now works on burden_report objects, returning the same verdict line as print() and summary().

  • New realised_exposure() returns, for every respondent including those who broke off or were screened out, the blocks the survey flow routed them into and the GfS+ points, items and pages they were shown in each block. Branch logic and display logic are evaluated against each respondent’s own answers, in flow order. Logic the data cannot settle (an unsupported literal, or an embedded field absent from the export, such as one set by question JavaScript) falls back to whether the respondent answered, and is counted per respondent rather than hidden.

  • New exposure_person_period() turns realised_exposure() output into person-period data for a discrete-time hazard model of break-off: the blocks each respondent was at risk in, the burden shown before each, and the block’s design score.

  • Display-logic groups joined by “And If” are now evaluated before groups joined by “Else If”: If A / Else If B / And If C is A or (B and C), not (A or B) and C. On a large Qualtrics export this reading matched which questions respondents answered (99.7%, against 60% for left to right). This changes burden profiles for surveys that mix the two; the demo survey’s 75th percentile moves from 129 to 130 points.

surveyBurden 0.2.0

New features

  • New realised_burden() scores the questions each respondent actually answered, using exported response data. Case-insensitive ExportTag matching.
  • New validate_catalogue() checks and coerces a hand-built question catalogue, enabling platform-agnostic scoring without a QSF file.
  • New “Extensions” vignette showing the catalogue schema and a worked example of scoring a non-Qualtrics survey.
  • New “Realised burden” vignette with a Qualtrics-export walkthrough.
  • gfs_weights() renamed to gfs_scheme(), and the weights argument of the scoring and reporting functions renamed to scheme. The old names have been removed, not deprecated: update calls to gfs_scheme() and scheme =.
  • burden_report() gains a words_per_line argument for per-respondent reading-load estimation.
  • burden_report() gains a max_paths argument (default 10,000), passed to resolve_flow(), so surveys with heavily branching flows can be analysed by raising the cap. The “too many paths” error now suggests it.
  • When routes is supplied, burden_report() now includes the arithmetic mean and the 10th and 90th percentiles of per-respondent predictions in $population and its printed table, alongside the existing five statistics.
  • Scoring and documentation now use “GfS+” to distinguish the package’s extended scheme (with documented inference rules for dropdowns, sliders, and multi-answer matrices) from the original GfS table.

Performance

Bug fixes

  • parse_qsf() no longer fails on sliders whose tick labels Qualtrics stores as numbers rather than text.
  • Surveys whose flow sits inside an Authenticator node are now resolved; previously the node and everything in it was skipped, giving an empty path and a burden of 0.
  • A Branch, Group or BlockRandomizer flow node with no child flow no longer crashes resolve_flow(); it is treated as empty.
  • Vignettes updated from weights = to scheme = after the rename.
  • realised_burden() now reads a Finished column coded TRUE/FALSE or yes/no (any case) as well as 1/0. Blank or unrecognised values are NA (status unknown); previously TRUE/FALSE became NA, and values such as "2" or "0.5" were misread as finished or not finished.
  • respondent_burden() (and burden_report(routes = ...)) now sets matched = TRUE only when a respondent’s optional-block set matches an enumerated path exactly. Routes that fall back to the path with no optional blocks were previously reported as matched; they are now matched = FALSE and the warning says which fallback path each group received. Predictions are unchanged.
  • realised_burden(): columns mapped through a col_map attribute now keep their Loop & Merge iteration, from an N_ column-name prefix or from a list(qid = , iteration = ) entry. Previously every mapped column was treated as iteration 0, so answers from different iterations of a looped question were merged and scored once. The help and vignette now also state that col_map overrides automatic column matching.
  • realised_burden() removes the question-label and ImportId rows at the top of a raw Qualtrics CSV export, with a message, when it can recognise them. Previously they were scored as respondents. Unrecognised rows are always kept.
  • validate_times() reads Qualtrics’ Duration (in seconds) column directly (as exported, or as read.csv() renames it), converts text durations to numbers, removes a raw export’s header rows, and gives a clear error when no duration column is found. Previously only completion_mins / completion_seconds were recognised, and a missing column failed with an unhelpful message.
  • Display-logic reachability: a question is now classed as never shown on a path only when its logic is shown to be false there. Previously any reference to a question off the path ruled it out, which was wrong for OR conditions, ElseIf groups and negated conditions (“not selected”, “not displayed”), and understated that path’s burden. Questions that depend on a ruled-out question are now re-checked, and enumeration fixes off-path trigger questions as unanswered. On the surveys tested the burden figures are unchanged; the fix matters for surveys with such logic across branches.
  • classify_reachability() gains a display_logic argument (the questions’ DisplayLogic trees). Without it the logic cannot be evaluated, so it no longer classes any question as unreachable; resolve_paths() supplies the trees.

Documentation

  • Interpretation wording corrected throughout. The structural profile was described as counting each combination “once” with unweighted quantiles; in fact each modelled state carries an equal model weight, each path’s weights sum to 1, and the report gives each complete path equal total weight. “Feasible” / “achievable” paths are now described as the paths the flow allows, with a note that branch conditions are not checked against each other. “Exact” is qualified: the printed certainty lines now read “have no unresolved display-logic question” and “enumerated in full within the model”, and ?calculation_certainty explains that neither means the result matches the live survey. The index column is documented as points / rare_threshold (it can exceed 1).
  • realised_burden() now has an explicit operational definition: a question’s full score counts when at least one mapped response column is non-blank, per detected loop iteration. It does not measure exposure, reading time or partial completion, and n_questions_answered counts question-iteration pairs. The vignette no longer presents a predicted-minus-realised gap as evidence of dropout, and explains that predictions use each respondent’s inferred route and loop counts.
  • The scoring vignette now lists the automatic rule for every question type and which gfs_scheme() entries each uses. Nine Table 1 entries (rank_per_item, open_medium, please_specify, open_essay_per_extra, filter, branching, sc_2_alt, sc_3_alt, sc_per_variable) are reference only – no rule reads them, so changing them does not change scores; ?gfs_scheme says so too. It also states that item scores are per iteration, that loop profiles run from 1 to the cap, and what can make the reported figures too low.
  • The paths vignette has a runnable recipe for building visit_* and loop_* route columns from a response export, and for checking matched before summarising, with what a match does and does not certify. ?respondent_burden and the vignette now state how 0, NA, over-cap and non-logical values in route columns are read.
  • The calibration vignette has a runnable validate_times() recipe: filtering non-finishers first, the accepted duration columns, adding route columns, choosing and reporting the trim, and how to read ratio, cor and implied_points_per_minute.
  • The extensions vignette and README no longer say downstream functions work “unchanged” on any catalogue. They now separate what works now (score_burden() on a catalogue), what needs an adapter (path_burden() with a hand-built paths table, shown runnable) and what still needs a Qualtrics survey (profiles, burden_report(), respondent functions), and note that has_display_logic = FALSE removes conditionality rather than skipping the profile.
  • New “Revising a survey to reduce burden” vignette: find the heaviest blocks and questions, check how they were scored, make a change, re-score and compare, and what the comparison does and does not establish. README and the get-started vignette open with a “where to start” table.
  • Every guide now opens with what it answers, what you need and what it cannot establish, and ends with a “Next” section. The website lists the guides in reading order.
  • “Path” now always means a way through the survey its design allows, and “route” what one respondent actually did; both are in the vocabulary table, which is now also in the README.
  • Added runnable @examples to all exported functions using the shipped demo fixture (demo_travel_survey.qsf).
  • Switched burden_report() and calculation_certainty() examples from \dontrun{} to \donttest{} with the demo fixture.
  • Added the Heimgartner and Axhausen (2024) DOI to the DESCRIPTION and to the documentation of gfs_scheme(), score_burden() and burden_report().
  • Updated .Rbuildignore to exclude manuscript and temporary files.
  • Updated roxygen2 to 8.1.0.

surveyBurden 0.1.1

Patch: corrected the documentation of respondent_burden() (it predicts per-respondent burden from loop counts and branch visits, then takes quantiles — not a route-frequency-weighted average). Rewrote the population-weighted burden section of the paths-and-display-logic vignette to match. Added software_repository_url to the JOSS paper frontmatter and expanded the Statement of Need to meet the 250-word guideline.

surveyBurden 0.1.0

First release.

  • burden_report() scores a Qualtrics .qsf (or a live survey via the API) with the GfS+ burden-point scheme (extending Heimgartner and Axhausen 2024), reconstructs the respondent paths the survey’s flow and display logic allow, and reports burden across those paths as a structured object.
  • print() on the report opens with a one-line verdict: the median completing-path burden in points and minutes, its ratio to the 399-point GfS+ benchmark, and the range across completing paths. summary() gives the same headline.
  • Supplying observed respondent routes turns the structural profile into a population-weighted burden figure.
  • burden_report(quiet = FALSE) reports progress through the pipeline stages with cli::cli_progress_step().
  • A survey where every structural path screens out (an unconditional EndSurvey in the flow) returns a report with NA burden and an explanatory warning rather than erroring; the per-path burdens are still in $paths.
  • Lower-level steps are exported for inspection or reuse: read_qsf(), fetch_qsf(), parse_qsf(), classify_question(), gfs_scheme(), score_burden(), resolve_flow(), resolve_paths(), path_burden(), path_burden_profile(), respondent_burden(), calculation_certainty(), validate_times().
  • Five vignettes: “Get started”, “Reading the report”, “The GfS+ scoring method”, “Paths and display logic”, and “Calibration and limits”.