20 · Interoperability
Purpose: Specify how Rasd Forms exchanges forms and data with the rest of the humanitarian data ecosystem — XLSForm/ODK XForms import and export, SurveyJS import, KoboToolbox / ODK Central / Ona integration, M&E export formats, webhooks, analytics feeds, data dictionaries, catalog metadata and the conformance corpus that proves it.
Audience: Engineers building @rasd/xlsform, @rasd/sync (openrosa transport), @rasd/server exports and @rasd/cli; developers at UN/NGO organisations migrating existing XLSForms or wiring Rasd into Kobo/Central/Power BI/DHIS2 pipelines.
TL;DR
- XLSForm is the sector's lingua franca (research/01 §5);
@rasd/xlsform(Apache-2.0) imports XLSForm/XForm → RFD and exports RFD → XLSForm that pyxform 4.5.0 compiles (research/14 §3, §6). - Import is best-effort, never lossy: every unmapped column, sheet, appearance or XPath fragment is preserved under the reserved
extkeysorg.getodk.xlsform,org.getodk.xpath,org.kobotoolboxwith a cell-level warning; export re-emits raw text when the REL AST is unchanged, so an unchanged import → export is byte-comparable in pyxform output. - ~97 % of real-world expressions map 1:1 to REL; the four true gaps (anchored
regex, string booleans,NaNcoercion, node-set semantics) are handled byregex(…, 'full'), thexpathcoercion hint, and${rep[].q}/countIf()rewrites. - One XForm-instance serializer/parser in
@rasd/xlsformpowers theopenrosasync transport (profileskobo,central,ona,generic) and the server-side bridge pattern;meta/instanceID = uuid:<submissionId>is the universal idempotency key. @rasd/serverexports Kobo/Ona-family CSV/XLSX by default with amode=centralswitch, plus JSON/JSONL, GeoJSON and a media zip; a Central-shaped OData 4.0 feed lets existing Power BI templates work unchanged.- Webhooks (
X-Rasd-SignatureHMAC-SHA256), data dictionaries (generateDataDictionary()), Dublin-Core-style catalog metadata and a public conformance dashboard round out the story. - Nothing here depends on the license state: import/export/
storage.export()always work (spine P6).
1. Scope and principles
| Direction | Package | Phase (19 · Roadmap) | Notes |
|---|---|---|---|
XLSForm (.xlsx) → RFD | @rasd/xlsform importXlsform() | 2 (E2.5) | Also XForm XML → RFD (importXform()) for forms pulled from servers |
| RFD → XLSForm | @rasd/xlsform exportXlsform() | 2 (E2.5) | Targets generic (pyxform), kobo, central |
| SurveyJS JSON → RFD | @rasd/xlsform importSurveyJs() | 2 (C, best-effort) | FR-135 (W); no export |
| Submission ⇄ XForm instance | @rasd/xlsform serializeXformInstance() / parseXformInstance() | 2 (E2.5; consumed in 3) | Used by the openrosa transport (E3.5) and by server-side bridges |
| RSP ⇄ OpenRosa/Kobo/Central/Ona | @rasd/sync createOpenRosaTransport() | 3 (E3.5) | 10 · Sync protocol §10 |
| Data exports, OData, webhooks | @rasd/server | 2–3 (FR-133 S/C) | 10 · Sync protocol §9.5 |
Principles: (1) fidelity over prettiness — keep the author's names, order, languages and raw expressions; (2) warn, don't fail — an import yields a valid RFD plus issues with sheet/row/column coordinates, never a half-document; (3) no JS XLSForm→XForm compiler — none is maintained (xls2xform 2016, xform-to-json 2017); Node helpers and CI shell out to pyxform (research/14 §1); (4) the runtime never needs XLSForm — conversion is an authoring-time step or a phase-3 transport concern.
// @rasd/xlsform — public API. Names follow [17 · API reference] §12 (the canonical export list); the option
// bags below are the interop-level detail that 17 summarises. Nothing here is part of the spine §11 frozen surface.
importXlsform(input: ArrayBuffer | Uint8Array | Blob | XlsformWorkbook,
opts?: { id?: string; version?: string; defaultLocale?: string; // overrides for missing/invalid settings cells (17 §12)
locales?: 'header' | 'code'; nestGroups?: boolean; coercion?: 'rel' | 'xpath'; kobo?: boolean; strict?: boolean }):
Promise<{ definition: FormDefinition; warnings: ImportIssue[]; report: ImportReport }>; // rejects with RasdError RASD_XLSFORM_IMPORT on structural errors (details.sheet/row/column)
importXform(xml: string, opts?: { coercion?: 'rel' | 'xpath' }): Promise<{ definition: FormDefinition; warnings: ImportIssue[]; xform: XformInfo }>;
exportXlsform(def: FormDefinition, opts?: { target?: 'generic' | 'kobo' | 'central'; locales?: 'header' | 'code'; format?: 'xlsx' | 'csv-zip' }):
Promise<{ bytes: Uint8Array; warnings: ImportIssue[] }>; // rejects with RASD_XLSFORM_EXPORT
serializeXformInstance(sub: Submission, def: FormDefinition, xform: XformInfo, opts?: { emitEmpty?: boolean; deprecatedId?: string }): { xml: string; media: MediaPart[] };
parseXformInstance(xml: string, def: FormDefinition, xform: XformInfo): { data: Record<string, unknown>; attachments: Partial<Submission['attachments']>; warnings: ImportIssue[] };
importSurveyJs(json: unknown, opts?: { defaultLocale?: string }): Promise<{ definition: FormDefinition; warnings: ImportIssue[] }>;
xpathToRel(xpath: string, ctx?: { scopePath?: string }): { rel: string; lossless: boolean }; // throws RASD_XLSFORM_UNSUPPORTED
relToXpath(rel: string): { xpath: string; lossless: boolean };
interface ImportIssue { code: string; severity: 'error' | 'warning' | 'info'; sheet?: string; row?: number; column?: string; path?: string; message: string; original?: string }
// XformInfo = { rootName, id, version, formhubUuid?, nodeOrder: string[], paths: Record<elementName, xpath>, metaNs?: 'orx' | null } — recorded at import, required by the serializer (research/14 §2 rule 0)
CLI: rasd convert xlsform in.xlsx -o out.form.json [--strict] [--kobo] [--json] and rasd convert rfd out.form.json --to xlsform --target kobo [--format xlsx|csv-zip]; exit code 1 on errors, 0 with warnings; --strict promotes W_XPATH_UNMAPPED and W_FUNCTION_UNKNOWN to errors for CI gates (exit 2, 17 · API reference §13). Import/export never consult the license state (spine §9: export of data and definitions always works).
2. XLSForm / XForm → RFD
flowchart LR
A["xlsx or XForm XML"] --> B["Sheet reader<br/>survey, choices, settings, entities, extra"]
B --> C["Structural pass<br/>groups, repeats, pages, types"]
C --> D["Expression pass<br/>XPath to REL, scope-aware rewrites"]
D --> E["Choice pass<br/>lists, cascades, datasets"]
E --> F["Kobo fold<br/>score, rank, kobomatrix"]
F --> G["validateFormDefinition"]
G --> H["RFD + warnings + report"]
Every pass is pure and deterministic (same workbook + same options → same RFD and the same definitionHash; the importer never stamps meta.createdAt/updatedAt or uuid() values itself — the CLI or builder does), so imports are reproducible in CI and diffs between two imports of the same file are empty. Cell coordinates travel with every issue (sheet, row = 1-based Excel row, column = header text) so the builder can deep-link into the original workbook.
2.1 Settings sheet
XLSForm settings | RFD | Notes |
|---|---|---|
form_id | id | Must match the RFD slug rule ^[a-z0-9][a-z0-9_-]{0,63}$ (04 · Schema §13); otherwise lower-cased/slugified + W_ID_SLUGIFIED. The original form_id is always kept in ext["org.getodk.xlsform"].settings.form_id and .xform.id because Kobo asset uids are mixed-case (aXyZ…) and the XForm root id attribute must be reproduced byte-exactly on export and in every serialised instance |
version | version | Empty → "1" + W_VERSION_DEFAULTED; re-importing an already-published id/version pair yields a definition the server will refuse to publish again (spine §4.3b) — the report says so (I_VERSION_ALREADY_PUBLISHED) when the CLI is given --previous <form.json> |
form_title | meta.title | Localised if form_title::Lang columns exist |
default_language | settings.defaultLocale | "Arabic (ar)" → ar; bare names guessed from a table (W_LANG_CODE_GUESSED) |
instance_name | settings.instanceName (REL) | |
style | pages → settings.navigation: "paged"; theme-grid → ext | see §2.5 |
public_key | settings.encryption: { mode: "submission", publicKeyId: <SHA-256 fingerprint> } + raw key in ext["org.getodk.xlsform"].settings.public_key | ODK RSA envelope is produced only by the openrosa transport (research/14 §1.1) |
name (root node), namespaces, attribute::*, submission_url, auto_send, auto_delete, client_editable, allow_choice_duplicates, clean_text_values | ext["org.getodk.xlsform"].settings.*; root name also in .xform.rootName | Needed for byte-exact export and instance serialisation |
Kobo kobo--locking-profile, kobo--lock_all | ext["org.kobotoolbox"].locking | verbatim |
2.2 Survey sheet columns
| Column | RFD | Rule |
|---|---|---|
type, name | type, name | see §2.3; reserved names (meta, instanceID, …) → E_RESERVED_NAME unless they are ODK meta rows (dropped, see below) |
label[::Lang (code)], hint, guidance_hint, constraint_message, required_message, media::*::Lang | label, hint, guidance, constraintMessage, requiredMessage, media as localized strings keyed by BCP-47 | Original header strings kept in ext["org.getodk.xlsform"].languages ({ "ar": "Arabic (ar)" }) so export reproduces them; ${x} inside labels → Mini-Message {x} |
required | required: true | REL | yes/true() → true; anything else → REL |
relevant, constraint, calculation, choice_filter, repeat_count | relevant, constraint, calculate, props.choiceFilter, props.count | XPath → REL (§2.7); raw text in ext["org.getodk.xlsform"].raw.<column> |
default | default.value (typed by question type) or default.expr when the cell parses as REL with a call or ${} reference | ODK dynamic-default semantics = once()-like first-load; W_DEFAULT_DYNAMIC |
read_only | readonly | |
appearance | appearance.variant + appearance.ext["org.getodk.xlsform"].raw | §2.6 |
parameters (k=v pairs, space/comma separated) | typed props (start/end/step, max-pixels → maxPixels, quality, capture-accuracy → accuracyThreshold, warning-accuracy → warningThreshold, value/label for _from_file; allow-mock-accuracy → ext) | unknown keys → ext |
trigger | logic.triggers[] { id: "xls-trigger-<name>", when: "notEmpty(${src})", on: "change", actions: [{ type: "setValue", target: "<name>", expr: "<calculation>" }] } + W_TRIGGER_APPROX | XForms xforms-value-changed fires on every change of ${src}; the RFD trigger fires on the false→true edge only (04 · Schema §8.2; see open questions). A trigger on a row that also has calculation is the ODK "recalculate on change" idiom — the importer emits the trigger and drops the element's calculate (a calculate target is E_TRIGGER_TARGET_CALCULATED) |
save_to, entities sheet | ext["org.getodk.xlsform"].entities + W_ENTITIES_UNSUPPORTED | Records/entities integration is FR-134 (C) |
body::*, bind::*, instance::*, unknown columns | ext["org.getodk.xlsform"].body / .bind / .instance / .row[column] | verbatim; rowIndex stored for stable export order |
ODK meta rows (start, end, today, deviceid, username, email, phonenumber, subscriberid, simserial, audit) are not elements in RFD. The importer drops the row (recorded in ext["org.getodk.xlsform"].metaRows) and rewrites references: ${start} → ${meta.startedAt}, ${end} → ${meta.now} (W_META_END_APPROX: ODK's end is stamped at finalize; the serializer writes submission.meta.finalizedAt into the end node), ${today} → today(), ${deviceid} → ${meta.deviceId}, ${username} → ${meta.username}; email/phonenumber/subscriberid/simserial become hidden with default.expr: "${meta.custom.<name>}" (W_META_HOST_PROVIDED); audit → settings.audit (track-changes → trackChanges, location-* → settings.audit.location). All meta rows are re-emitted on export and serialised into the instance from submission.meta so Kobo/Central still receive start/end/deviceid columns.
2.3 Question types
| XLSForm type | RFD type (+ props) | Loss / warning |
|---|---|---|
text | text (multiline from appearance; format: "url" from url) | — |
integer / decimal | number kind: "integer" | "decimal" | — |
range | range (min/max/step from parameters start,end,step, defaults 0/10/1) | — |
select_one L / select_multiple L | select_one / select_multiple with props.list: "L"; or_other → props.other.enabled | — |
select_one_from_file f.csv/.xml/.geojson, select_multiple_from_file | same + choiceLists[L] = { source: { type: "dataset", dataset: "f" }, valueKey, labelKey } and datasets[] = { name: "f", source: "server", keyField } | Media file must be attached at deploy time; W_DATASET_SOURCE_ASSUMED |
select_one_external | select_one over dataset itemsets with choiceFilter from the external choices sheet | — |
rank L | rank | — |
note | note | — |
geopoint / geotrace / geoshape | same | — |
date / time / dateTime | date / time / datetime (calendar from appearance) | time offset dropped on import, re-added on serialise |
image (+ appearance signature) | image / signature; draw/annotate → image props.annotate: true | — |
audio / video / file | same | — |
background-audio | audio + ext["org.getodk.xlsform"].type = "background-audio" | W_TYPE_DOWNGRADED (no background capture in v1) |
barcode | barcode | — |
calculate | calculate | — |
acknowledge | checkbox | export writes OK/empty |
hidden | hidden | — |
xml-external / csv-external | datasets[] entry | — |
begin_group / end_group | group (transparent by default; opts.nestGroups → props.nestData: true) | XForm path recorded in ext["org.getodk.xlsform"].xform.path |
begin_repeat / end_repeat | repeat (count from repeat_count) | — |
Kobo begin_score … | group appearance.variant: "field-list" of select_one variant: "likert" + ext["org.kobotoolbox"].score | folds back on target: 'kobo' |
Kobo begin_rank … | rank + ext["org.kobotoolbox"].rank | folds back |
Kobo begin_kobomatrix … | group of per-row groups (children {item}_{q}) + ext["org.kobotoolbox"].matrix | v1 matrix is single-column (04 · Schema open question); W_KOBOMATRIX_EXPANDED |
| unknown type | non-strict: hidden + ext["org.getodk.xlsform"].type + W_TYPE_UNKNOWN; --strict: E_XLSFORM_TYPE_UNKNOWN | the value column survives round-trip; the renderer never sees the row |
2.4 Choices sheet, cascades and choice_filter
list_name, name, label[::Lang], media::image, geometry+ any extra column →choiceLists[list_name].choices[] = { value: name, label, media, attrs: { <extra columns> } }. Choicevaluemust be whitespace-free (E_CHOICE_VALUE_SPACE, since XForm joins multi-selects with spaces).- Cascading selects:
choice_filterstrings such asgov_code = ${gov}parse unchanged — bare identifiers resolve to the candidate choice'sattrs(05 · Logic);current()/../q→${q}. Lists > 500 choices are converted to an inline dataset (datasets[] = { name: "<list>_choices", source: "inline", keyField: "_rid" },choiceLists[list] = { source: {…}, valueKey: "name", labelKey: "label", filterKeys: [<attrs used with =>] }) so storage-side filtering applies (W_CHOICES_AS_DATASET). - Duplicate names (
allow_choice_duplicates=yes, common in admin cascades) violate RFD's unique-valuerule; the importer always converts such lists to the dataset form with a synthetic_ridkey and warnsW_CHOICE_DUPLICATES_AS_DATASET. jr:choice-name(${q}, '${q}')→choiceLabel(${q}, 'L')withLresolved from the referenced element.
2.5 Groups, pages and repeats
style: pages→settings.navigation: "paged"; every top-levelfield-listgroup becomes apage(id= group name); top-level questions outside such groups each get their own page (Collect's one-screen-per-question behaviour). Withoutpages→navigation: "scroll", single pagemain. Nestedfield-listgroups staygroupvariant: "field-list".- Groups are transparent (spine §4.3) but the XForm path (
/data/hh/name) is recorded so serialisation andgroup/questionexport headers are reproducible. - Repeats:
${q}inside its own repeat resolves to the current instance; a reference to a repeat child from outside the repeat inside an aggregate (sum(${age}),count(${hh})) is rewritten tosum(${hh[].age})(I_NODESET_REWRITE);position(..)→position();indexed-repeat(${q}, ${hh}, 2)is accepted as-is (alias);jr:templateis never emitted.
2.6 Appearance mapping
Collect ignores unknown appearances, so unmapped tokens are kept verbatim and are safe on export.
| XLSForm appearance | RFD |
|---|---|
minimal | select_* variant: "dropdown" |
quick, quickcompact, compact, columns, columns-n, columns-pack | variant: "buttons" (+ columns: n); quick (auto-advance to the next question) has no RFD v1 prop — kept as props.ext["org.getodk.xlsform"].autoAdvance: true (SelectOneProps is closed, additionalProperties: false); candidate props.autoAdvance for RFD 1.1 (open questions) |
likert | variant: "likert" |
autocomplete (search) | props.search: true |
label, list-nolabel, image-map, map, placement-map, no-buttons, hidden-answer | verbatim in appearance.ext["org.getodk.xlsform"].raw (W_APPEARANCE_UNMAPPED, info) |
field-list, table-list | group.appearance.variant: "field-list" (table-list adds ext) |
multiline / numbers / url / masked / thousands-sep | text.props.multiline / number hint / format: "url" / mask / thousandsSeparator |
signature / draw / annotate / new / new-front / selfie | signature type / annotate: true / source: "camera" |
no-calendar, month-year, year, islamic, ethiopian, coptic, bikram-sambat, … | date.props.calendar: "hijri" for islamic; others verbatim |
bearing, vertical, distress, rating, counter, printer, ex:* | verbatim; ex:* → x:odk-intent placeholder + W_INTENT_UNSUPPORTED |
2.7 XPath → REL
The mapping table lives in 05 · Logic §16; the importer's contract:
| Case | Action | Code |
|---|---|---|
${q}, ../q, /data/g/q, ., current()/../q | ${q}, ${../q}, ${/q} (via path map), ., ${q} | — |
ODK function names in call position (string-length(, selected-at(, count-selected(, indexed-repeat(, format-date(, date-diff(), operators div, mod, and, or, not(, literals true(), false() | accepted unchanged — these are the only hyphenated aliases the REL parser knows (spine §5); everything else hyphenated is rewritten (rows below) | — |
jr:choice-name(${q}, '${q}') | choiceLabel(${q}, 'L') (L resolved from the referenced element; the jr: prefix is not REL) | I_EXPR_REWRITTEN |
instance('L')/root/item[name=${q}]/label | choiceLabel(${q}, 'L') | I_EXPR_REWRITTEN |
instance('csv')/root/item[k=${x}]/v, pulldata('csv','v','k',${x}) | pulldata('csv','v','k',${x}) | — |
count(/data/rep[q='yes']) | countIf(${rep}, ${q} = 'yes') | I_EXPR_REWRITTEN |
decimal-date-time(a) - decimal-date-time(b), int((today() - ${dob}) div 365.25) | dateDiff(a, b, 'days'), age(${dob}) | I_EXPR_REWRITTEN |
regex(., 'p') | regex(., 'p') — REL default mode 'full' = JavaRosa Pattern.matches; pattern lacking ^…$ and form origin Enketo/Kobo web → info | W_REGEX_UNANCHORED |
| Empty-string arithmetic / string booleans | settings.ext["org.getodk.xpath"].coercion = "xpath" hint (opts.coercion) | I_COERCION_HINT |
Functions outside the REL core (boolean-from-string, checklist, weighted-checklist, decimal-date-time, decimal-time, log10, digest, randomize, geofence, intersects, format-date-time, …) | hyphenated names are rewritten to the REL identifier form (boolean-from-string( → booleanFromString(, decimal-date-time( → decimalDateTime(, format-date-time( → formatDateTime() so the expression parses; the function is still unknown at load (RASD_EXPR_UNKNOWN_FUNCTION — the form refuses to open, 05 · Logic §8) unless the host registers the opt-in compat pack: registerOdkFunctions() from @rasd/xlsform/rel-odk (camelCase names, pure per function, digest needs WebCrypto/RN crypto) and validates with validateFormDefinition(def, { functions: odkFunctionNames }). The report lists every such function per element | W_FUNCTION_UNKNOWN (error with --strict) |
once(expr) in relevant/constraint/required (only meaningful in calculate and default.expr, 05 · Logic §7.7) | inner expression kept, once() wrapper dropped; in default the wrapper is dropped too because default.expr already has first-load semantics | W_ONCE_MISPLACED / I_EXPR_REWRITTEN |
Axes (ancestor::), //, *, union |, name()/lang()/id(), positional predicates other than indexed-repeat, instance() node-sets fed to aggregates, dynamic jr:itext(concat(…)) | property left empty, verbatim under ext["org.getodk.xpath"].<property> | W_XPATH_UNMAPPED (error with --strict) |
An unmapped relevant means the element is always shown; an unmapped constraint is not enforced — the report lists these explicitly so reviewers can decide before deployment.
2.8 Media
media::image|audio|video[::Lang] and jr://images|audio|video|file|file-csv/… → media.image/audio/video by bare filename (assets/<file>); CSV/XML/GeoJSON jr://file-csv/ refs become datasets. Media files themselves are not inside the workbook; the report lists expected filenames so the host (or the OpenRosa manifest) can supply them.
3. RFD → XLSForm export
Rules, in order: (1) if the REL AST hash of a property equals the hash recorded at import, re-emit ext["org.getodk.xlsform"].raw.<column> verbatim; otherwise print REL → XPath with relToXpath() (stringLength( → string-length(, ${rep[].q} → ${q} inside the aggregate, countIf(${rep}, p) → count(${rep}[p]) with ${x} inside p printed relative to the repeat, choiceLabel(${q}, 'L') → jr:choice-name(${q}, '${q}'), ? : → if(), == → =, &&/||/! → and/or/not(), ${meta.deviceId} → ${deviceid} when the meta row is re-emitted); REL that has no XPath equivalent (jsonPath, ${obj.key} member access, host functions, pulldata on an inline dataset without a file) is emitted as a calculation placeholder with W_REL_UNEXPORTABLE; (2) Rasd-only types degrade with W_EXPORT_DOWNGRADED: rating → select_one + likert appearance, checkbox → acknowledge, consent → note + acknowledge (+ text_version calculate), matrix → group of select_one (Kobo target: begin_kobomatrix when ext["org.kobotoolbox"].matrix exists), text format: email → text + regex constraint, x:* → text + ex: appearance; (3) pages → style: pages + field-list groups; transparent groups export as begin_group (data nesting differs — stated in the report); (4) validators[] fold into one constraint joined with and; severity: warning validators are dropped with a warning; (5) languages use recorded header strings (label::Arabic (ar)), else label::<code>; (6) datasets → select_*_from_file (source: server|url) or inline choices; (7) extraSheets (osm, kobo--locking-profiles, external_choices) and column order are restored; (8) settings.audit → audit row; (9) logic.calculated[] → calculate rows.
target: 'kobo' refolds org.kobotoolbox constructs and emits _or_other; target: 'central' keeps standard XLSForm. Node/CI runs pyxform (xls2xform) after export and stores the status in report.pyxform (code 100/101/999).
4. SurveyJS JSON import (best-effort)
SurveyJS (survey-core 3.0.0, MIT) has the largest web installed base (research/02 §1). Import is one-way and clean-room (schema mapping only).
| SurveyJS | RFD | Notes |
|---|---|---|
pages[], panel | pages[], group (nestData: false) | pages never nest in either |
paneldynamic (templateElements, minPanelCount, maxPanelCount, keyName, confirmDelete) | repeat (elements, min, max, keyField, confirmDelete) | displayMode ignored |
text (inputType: number/email/date/…) | number / text format / date | |
radiogroup/dropdown, checkbox, tagbox, ranking, rating, boolean, signaturepad, file, html, expression, matrix (single-choice rows) | select_one, select_multiple, rank, rating, checkbox, signature, file, note, calculate, matrix | matrixdropdown/matrixdynamic → repeat of groups + W_SURVEYJS_APPROX |
choices[] (value, text), showOtherItem, showNoneItem, choicesByUrl | choiceLists, props.other, exclusive: ["none"], datasets[] source: "url" (offline-cached) | choicesVisibleIf → choiceFilter when expressible |
visibleIf, enableIf, requiredIf, isRequired, defaultValue, defaultValueExpression, setValueExpression, validators[] (numeric, text, regex, email, expression, answercount; notificationType) | relevant, readonly (negated), required, default, calculate, validators[] (range, length, regex, expr; notificationType → severity), email → text format: "email", answercount → props.minSelected/maxSelected | expression DSL → REL: {q} → ${q}, {panel[0].q} → ${panel[1].q} (0- → 1-based), {panel[-1].q} → indexedRepeat(${q}, ${panel}, count(${panel})), iif → if, notempty/empty → notEmpty()/empty(), contains/anyof/allof → selected() forms, <> → !=, {$question.visible} and {matrix.row.col} → unmappable; unmappable → ext["com.surveyjs"].expr + W_SURVEYJS_EXPR_UNMAPPED (property left empty, listed in the report) |
triggers[] (complete, setvalue, copyvalue, skip, runexpression), calculatedValues[] | logic.triggers[] (complete → complete, setvalue/copyvalue/runexpression → setValue, skip → skipTo when gotoName is a page or the first element of one, else W_SURVEYJS_APPROX), logic.calculated[] (includeIntoResult → includeInData) | trigger expression → when (edge-triggered in RFD, 04 · Schema §8.2) |
{ default, ar } localized strings, locale | { <defaultLocale>, ar }, settings.defaultLocale | default key renamed to the survey locale or en |
everything else (showProgressBar, completedHtml, themes) | ext["com.surveyjs"] | preserved, not rendered |
5. KoboToolbox / ODK Central / Ona integration patterns
Two patterns; pick per deployment (research/14 §5, 10 · Sync §10).
5.1 Pattern A — openrosa transport on the device (phase 3)
import { createSyncEngine } from '@rasd/sync';
import { createOpenRosaTransport } from '@rasd/sync/openrosa'; // sub-path so the RSP-only bundle never pays for the XML serializer
// Auth stays delegated to the host (spine §8): getAuthToken() returns the secret; `auth.kind` only says where the
// transport puts it — Kobo Data-Collector / Central app-user token in the URL path, `Authorization: Token …`,
// Basic, Digest (RFC 7616, generic OpenRosa) or Bearer. Never bake a token into the client bundle.
const getAuthToken = async () => host.enrolment.collectorToken(); // e.g. scanned from a Collect-style QR code, kept in storage.kv
const transport = createOpenRosaTransport({
profile: 'kobo', // 'kobo' | 'central' | 'ona' | 'generic'
baseUrl: 'https://kc.kobotoolbox.org',
auth: { kind: 'token-path' }, // or 'api-token' | 'basic' | 'digest' | 'bearer'
getAuthToken,
jsonFastPath: false, // Kobo/Ona only: POST JSON instead of multipart
});
const sync = createSyncEngine({ storage, transport, getAuthToken, policy }); // baseUrl is unnecessary with a custom transport
17 · API reference §7 lists the minimal experimental signature ({ baseUrl, profile, getAuthToken, fetch? }); the generic profile and the auth/jsonFastPath options above are this document's phase-3 design for it and land in 17 when the transport stabilises.
| Engine operation | Transport behaviour |
|---|---|
pullForms() | GET …/formList (X-OpenRosa-Version: 1.0); hash (md5:) mapped onto definitionHash; profile URL builders: Central /v1/key/{token}/projects/{id}/formList, Kobo /collector/{token}/formList or /{username}/formList |
fetchForm(id, version) | download XForm XML → importXform() → RFD; manifest mediaFiles → media/dataset refs; type="entityList" / entities.csv (ETag) → datasets[] |
pushSubmissions(batch) | per submission: serializeXformInstance() (root, order, formhub/uuid from the pulled XForm), HEAD …/submission → X-OpenRosa-Accept-Content-Length (Kobo default 10,000,000 B; Central 100 MB), multipart POST with xml_submission_file + media, chunked by size with the XML repeated in every chunk; Central profile may instead POST …/forms/{xmlFormId}/submissions (XML body) + per-file POST …/attachments/{filename} |
attachments.* | the transport reports capabilities().attachments === 'inline' (an optional method proposed for SyncTransport, 03 · Architecture §8) → the engine skips the tus phase and marks attachments uploaded when the multipart chunk that carried them is acknowledged; media travel inside the multipart |
registerDevice() | no-op returning default policy; Collect-style QR settings (zlib+base64 JSON with general.server_url) can seed baseUrl/token |
records, events, wipe, license header | unavailable |
sequenceDiagram
participant E as Sync engine
participant T as openrosa transport
participant S as Kobo / Central
E->>T: pushSubmissions(batch, idempotencyKey)
T->>T: serializeXformInstance(sub) gives XML + media parts
T->>S: HEAD /submission
S-->>T: 204 + X-OpenRosa-Accept-Content-Length
loop one POST per chunk, each within the accepted length
T->>S: POST multipart (xml_submission_file + media)
S-->>T: 201 / 202 / 409 / 413
end
T-->>E: BatchResult per item: accepted, duplicate, conflict or rejected
Status mapping: 201/202 → accepted (Kobo 202 Duplicate Instance → duplicate); 409 with identical local hash → duplicate, otherwise conflict; 413 → halve chunk, else rejected {code:'too_large'}; 400 (bad XML / unknown attachment name on Central) → rejected; 404 → rejected {code:'form_closed'}; 401/403 → RASD_SYNC_AUTH (queue paused, data kept); 5xx/network → retry with the engine's backoff. Retries are safe because meta/instanceID = uuid:<submissionId> is the idempotency key on all four servers.
Blockers to verify early: CORS on kc.kobotoolbox.org / api.ona.io / Central for browser PWAs (unverified) — plan the relay transport (@rasd/server /v1/relay/openrosa/* proxying with the user's token); Digest auth in React Native (implement RFC 7616 MD5 for generic OpenRosa; prefer token-in-path/API tokens elsewhere).
5.2 Pattern B — RSP on the device, bridge on the server (host code)
Devices keep speaking RSP (resumable tus, quarantine, records); a bridge worker subscribes to the submission.completed webhook (§7), calls serializeXformInstance() and posts to Kobo/Central/Ona with a service token. Prerequisite: the same form is published there (exportXlsform({ target }) → Central POST /v1/projects/{id}/forms?publish=true, or a Kobo kpi asset import). One device protocol, no CORS/Digest on device, retries and audit in one place; the bridge writes the remote instanceID/_id back to ext["org.getodk.openrosa"].remote.
5.3 Pulling data back
Kobo kpi GET /api/v2/assets/{uid}/data/ (flat group/question keys, page ≤ 1000), Ona GET /api/v1/data/{pk} and Central OData ($expand=*) map through parseXformInstance()/a JSON mapper into Submission, with server system fields (_id, _uuid, _submission_time, _validation_status / __id, __system) under submission.ext["org.getodk.openrosa"].server.
6. Export formats for M&E teams
GET /v1/export/forms/{id}/submissions.{csv,xlsx,json,jsonl,geojson,zip} (admin token; a non-device route of the reference server, 10 · Sync protocol §2.10) with mode=kobo|central (default kobo), versions=all|latest|<v> (default all: union of columns + alias merge, always __version + __definitionHash), labels=names|labels&locale=ar, groupSep=/|-, multiSelect=both|summary|details, since=, status=, includeQuarantined=false, mediaUrls=true, redact=none|drop|hash (columns of bind.sensitive elements are dropped or replaced by a salted SHA-256; default none — the export audit log records who exported unredacted PII, 16 · Security). Exports > 50,000 rows or with media run as jobs (POST /v1/export/jobs → 202, poll, result URL valid 24 h). On device, storage.export() (JSONL + blobs) is always available; @rasd/xlsform exposes flattenSubmission(sub, def, { mode }) so CLI and hosts produce identical rows.
| Format | Shape | Notes |
|---|---|---|
| CSV (flat) | one row per submission; group/question headers; select_multiple as q (space-separated) + q/choice 0/1; geopoint q, _q_latitude, _q_longitude, _q_altitude, _q_precision; system columns _id, _uuid, _submission_time, _validation_status, _notes, _status, _submitted_by, __version__, _tags, _index | mode=central: group-question, KEY, SubmissionDate, SubmitterID, ReviewState, q-Latitude/-Longitude/-Altitude/-Accuracy; UTF-8 with BOM, RFC 4180, CRLF |
| CSV (repeat tables) | one file per repeat (<form>_<repeat>.csv) with _index, _parent_table_name, _parent_index (KEY/PARENT_KEY in central mode) | nested repeats chain parents |
| XLSX | same tables as sheets; ≤ 1,048,576 rows/sheet (Excel cap) else split _2, _3; header row frozen; RTL sheet direction when locale is RTL | xlsx streaming writer |
| JSON / JSONL | full Submission objects (data nested: repeats as arrays, groups per nestData), attachments as { id, filename, url } | JSONL is the archival format |
| GeoJSON | FeatureCollection; one Feature per geo answer (geoField= limits to one question; default = all geopoint/geotrace/geoshape); geometry Point/LineString/Polygon (WGS-84, [lng, lat, alt]); properties = flat row + __submissionId, __field, __repeatPath | Kobo _geolocation also emitted in JSON |
| Media zip | media/<submissionId>/<field>/<filename> + manifest.csv (submissionId, field, filename, mime, bytes, sha256) + submissions.csv | streamed; sha256 verified |
| Audit | audit.csv per submission (ODK columns event,node,start,end,old-value,new-value,user,change-reason) inside the zip | from submission.audit[] / audit.jsonl |
Cross-version alias merge follows research/12 §6: renamed columns merge via the server rename map (the MigrationPlan rename steps recorded at publish, spine §4.3b); retired choices export the stored value; per-row __version tells analysts which schema applies. Never silently skip an unknown version (formpack lesson) — quarantined rows are excluded by default but counted in the job report. Cell values are written with CSV/XLSX formula-injection protection: any text cell (free-text answers, labels, other text) starting with =, +, -, @, \t or \r is prefixed with ' (OWASP CSV-injection guidance); numeric, date and geo columns are typed and never escaped, so negative numbers stay numbers. rawCells=true disables the guard for admins who post-process the file.
7. Webhooks from @rasd/server
Events: submission.accepted, submission.completed (all attachments present — the one to trigger downstream pipelines), submission.quarantined, record.updated, record.conflict, form.published, device.registered. Body ≤ 256 KiB: attachments are referenced by signed URL (valid 1 h), never inlined; larger payloads set truncated: true and the receiver fetches GET /v1/submissions/{id}.
// POST <url> · headers: Content-Type: application/json · X-Rasd-Event: submission.completed
// X-Rasd-Delivery: 0198c1b0-… (idempotency) · X-Rasd-Signature: t=1755250000,v1=<hex HMAC-SHA256(secret, t + "." + body)>
{ "event": "submission.completed", "orgId": "org_unrwa", "occurredAt": "2026-08-15T10:03:11Z",
"submission": { "id": "0198c1a2-…", "formId": "pdm-gfd-2026", "formVersion": "3", "status": "synced",
"data": { "consent": "yes", "hh_members": [ { "name": "…", "age": 34 } ] },
"attachments": [ { "id": "…", "field": "photo", "mime": "image/jpeg", "url": "https://…?sig=…" } ] } }
Rules: verify the signature with a constant-time compare and reject |now − t| > 300 s; respond 2xx within 10 s and do the work asynchronously; retries back off 1 min → 24 h (max 10 attempts) then park for manual redelivery; ordering is not guaranteed — key on submission.id + clientRev; per-webhook events[] and optional formIds[] filters; 32-byte secrets rotatable with a 24 h dual-secret window; HTTPS only, private-network URLs refused (SSRF guard).
8. Downstream analytics
| Target | Path | Status |
|---|---|---|
| Power BI / Tableau / Excel | Central-shaped OData 4.0 minimal feed GET /v1/odata/forms/{id}.svc (Submissions, Submissions.{repeat}, __id, __system, $filter/$select/$expand=*/$top/$skip/$count/$skiptoken) so Central templates work unchanged; or scheduled CSV/XLSX | v1 (C in FR-133) |
| DHIS2 (Tracker/aggregate) | host bridge on submission.completed: element ↔ data element/attribute mapping in ext["org.hisp.dhis"], org unit from an admin select_one, POST to /api/tracker or /api/dataValueSets; option sets ↔ choice lists via the data dictionary | idea; sample bridge in apps/example-next |
| ActivityInfo | CSV import of the flat/repeat tables, or REST record push keyed by _uuid | idea |
| RapidPro (CFM/AAP loops) | webhook → flow start with non-bind.sensitive fields as contact fields | idea |
| Data warehouse / HDX | daily JSONL to object storage; HXL hashtag row from the dictionary for HDX uploads | idea |
9. Data dictionary generation
generateDataDictionary(def, { locale, mode: 'kobo' | 'central', include: ['choices', 'datasets'], previous?: FormDefinition[] }) in @rasd/xlsform — not @rasd/core, which stays inside its 45 kB budget (spine §12) and knows nothing about XLSForm types or export column conventions; the function only needs validateFormDefinition/diffDefinitions from core — (CLI rasd dictionary form.json --format csv|xlsx|md|json) produces one row per value-bearing element:
path (XForm path), name, page, repeat, type, valueType (JSON), label, hint, required, relevant, constraint, constraintMessage, list (choice list or dataset), choiceValues, exportColumns (per mode, e.g. q, q/yes, _q_latitude), xlsformType, sensitive (bind.sensitive), trackChanges, hxl (from ext["org.hxlstandard"]), extTags (host ext summary), introducedIn / retiredIn (from diffDefinitions across the previous versions). Appendices: choice lists (list, value, label::<locale>, attrs…), datasets (name, keyField, columns), triggers/calculated. The XLSX flavour is the codebook M&E teams attach to datasets and to the vendor pack; the Markdown flavour is what the Docusaurus site renders per published form. Labels are emitted in the requested locale with a fallback to settings.defaultLocale (W_DICT_MISSING_TRANSLATION per gap), and the XLSX sheet direction follows settings.localeMeta[locale].dir so Arabic codebooks read right-to-left.
10. Form catalog metadata (Dublin-Core-style)
RFD meta already carries title/description/tags/author/timestamps; catalog fields go in meta.ext["dev.rasd.catalog"] (v1; candidate for meta.catalog in RFD 1.1). @rasd/server exposes GET /v1/forms/{id}/catalog.json (also ?all=true for the org listing) using Dublin Core term keys:
| DC term | Source |
|---|---|
dc:identifier | urn:rasd:form:<orgId>:<id>:<version> (+ definitionHash) |
dc:title, dc:description, dc:subject | meta.title (all locales), meta.description, meta.tags |
dc:creator, dc:publisher, dc:contributor | meta.author, org name, catalog.contributors[] |
dcterms:created, dcterms:modified, dcterms:issued | meta.createdAt, meta.updatedAt, publish time |
dc:language | settings.locales |
dcterms:isVersionOf, dcterms:replaces | id, previous published version |
dcterms:conformsTo | https://schemas.rasd.dev/form/v1.json |
dc:rights, dcterms:license, dcterms:accessRights | catalog.rights, catalog.license (SPDX), sensitivity classification (`public |
dc:coverage (spatial/temporal) | catalog.coverage: { spatial: ["SY", "SY-DI"], temporal: { start, end } } |
| humanitarian | catalog.sector (cluster), catalog.activity, catalog.hxl |
11. Conformance and compatibility test corpus
| Corpus | Use | Licence |
|---|---|---|
pyxform 4.5.0 tests/fixtures/example_forms (36 files incl. xlsform_spec_test.xlsx, widgets.xlsx, pull_data.xlsx, choice_filter_test.xlsx, or_other.xlsx, repeat_date_test.xlsx, field-list.xlsx) + bug_example_forms (9) + XML goldens; ~90 unit-test markdown tables | XLSForm → XForm round-trip goldens | BSD-2 |
ODK Web Forms packages/common/src/fixtures/* + packages/scenario (JavaRosa-ported) | engine/XPath semantics | Apache-2.0 |
@getodk/xpath 1.0.0 test suite | REL-vs-XPath oracle | Apache-2.0 |
JavaRosa src/test/resources (~45 XForms), enketo-transformer test/forms (43), ODK Collect test forms | Collect/Enketo behaviour, appearances | Apache-2.0 |
| XLSForm.org examples; Kobo public library/templates; REACH/IMPACT MSNA, IOM DTM, UNHCR, WFP mVAM tools | realism: Arabic RTL, admin cascades, large repeats | mixed — catalogue in packages/xlsform/test/corpus/SOURCES.md; redistribution to be confirmed (02 · Requirements open question) |
CI jobs: (1) pyxform round-trip — XLSForm → pyxform → XForm A; XLSForm → RFD → XLSForm → pyxform → XForm B; canonicalise (sorted attributes, normalised whitespace, ignore version) and diff against an allow-list; (2) expression conformance — every corpus expression evaluated by REL (xpath mode) and @getodk/xpath over generated contexts, equal modulo the documented regex/NaN caveats; (3) submission goldens — fixture answers → serializeXformInstance() → validated against the pulled form's field list; nightly post to dockerised getodk/central and kobo-install, read back and diffed; (4) transport contract tests — recorded HTTP fixtures (msw) per profile; (5) coverage dashboard in the docs: % expressions parsed, % forms with zero warnings, per-function and per-appearance pass rates. Targets: ≥ 90 % of the ~140-form corpus imports with zero errors, ≥ 95 % of expressions parse unchanged (FR-022/130), 1,000-row import ≤ 3 s in Node (NFR-008).
12. Failure modes and how each surfaces
| Situation | Behaviour | Surfaced as |
|---|---|---|
Workbook has no survey sheet, unreadable .xlsx, or a type column missing | Import stops before any mapping — no half-document | RasdError RASD_XLSFORM_IMPORT (details.sheet/row/column); CLI exit 1 |
Row with unmappable XPath in relevant / constraint / calculation | Property left empty (element always shown / constraint not enforced / no calculate); verbatim text under ext["org.getodk.xpath"].<property> | W_XPATH_UNMAPPED per cell + a "logic dropped" summary at the top of the report; --strict → exit 2 |
Non-core ODK function (checklist, digest, …) | Expression parses; the form refuses to open at load until the compat pack is registered | W_FUNCTION_UNKNOWN at import; RASD_EXPR_UNKNOWN_FUNCTION at load (05 · Logic §8) |
Two elements resolve to the same name (case-insensitive, 04 · Schema §13 E_DUPLICATE_NAME — e.g. begin_kobomatrix expansion or transparent groups un-nesting hh/name and visit/name) | Second occurrence suffixed _2; original path kept in ext["org.getodk.xlsform"].xform.path | W_NAME_RENAMED (error with --strict); export restores the original from the recorded path |
form_id mixed-case / not a slug | Slugified id; original kept for the XForm root id | W_ID_SLUGIFIED |
| Choice list > 500 rows or with duplicate names | Converted to an inline dataset with _rid | W_CHOICES_AS_DATASET / W_CHOICE_DUPLICATES_AS_DATASET |
Media file referenced but absent (media::image, select_one_from_file) | Reference kept; report lists expected filenames | I_MEDIA_EXPECTED (host or manifest supplies the file) |
Export of a Rasd-only element type or severity: warning validator | Downgraded / dropped, never silently | W_EXPORT_DOWNGRADED; pyxform status stored in report.pyxform |
| Export of REL with no XPath equivalent | calculation placeholder | W_REL_UNEXPORTABLE |
| pyxform not installed / not on PATH in the Node helper | Export still succeeds; report.pyxform = { skipped: true } | I_PYXFORM_UNAVAILABLE (CI treats as failure, local dev as info) |
openrosa 413 after re-chunking down to a single attachment larger than the ACL | Submission stays in the outbox with rejected {code:'too_large'} and the offending attachment named | RASD_SYNC_REJECTED; UI shows the attachment so the enumerator can recapture at lower maxPixels |
openrosa 409 with different XML for the same instanceID (edited on the server or an earlier partial upload) | Not retried automatically | conflict status; user/supervisor resolves; nothing is overwritten |
openrosa 401/403 (token revoked, form permission removed) | Queue paused, data kept, no data loss | RASD_SYNC_AUTH → authRequired state; host prompts re-enrolment |
| Server export job exceeds 24 h result validity or is interrupted | Job re-runnable; partial files never served | GET /v1/export/jobs/{id} → status: "failed" + reason; admin UI |
| Webhook endpoint down > 24 h / 10 attempts | Parked for manual redelivery, nothing lost | delivery row parked; admin UI + GET /v1/webhooks/{id}/deliveries |
| Webhook secret rotated | 24 h dual-secret window; both signatures accepted | receiver verifies against v1 and, when present, v2 |
Kobo/Central form version differs from the pulled XformInfo (form redeployed server-side) | Serializer keeps emitting the version it was pulled with; server accepts old versions (research/14 §1.2) | formUpdated event after the next pullForms(); drafts migrate opt-in (spine §4.3b) |
The importer's report (ImportReport) is the contract with reviewers: { summary: { elements, pages, warnings, errors, logicDropped }, issues: ImportIssue[], expectedMedia: string[], languages: Record<code, header>, pyxform?: { code, message } }, and the builder renders it as a checklist before the form can be published (FR-130).
13. Security, privacy, performance and accessibility considerations
Untrusted input. An XLSForm or XForm arriving from a server, an e-mail attachment or a Kobo library is untrusted content (16 · Security): the .xlsx reader runs with size caps (default 25 MiB workbook, 50,000 rows, 5,000 columns; RASD_XLSFORM_IMPORT details.reason: "limit" beyond them) and no macro/external-link evaluation; the XForm/instance XML parser has DTD and external-entity resolution disabled (no XXE, no billion-laughs — a <!DOCTYPE is a hard error); jr:// and instance('…') references are resolved only against the manifest, never as file-system paths; imported expressions are REL source text that the engine parses without eval (spine §5). In the browser, import runs in a Web Worker so a pathological workbook cannot freeze the builder. The Node pyxform helper is CI-only, receives a temp-file path (never shell-interpolated), runs with a timeout and never on end-user devices.
Secrets and auth. OpenRosa/Kobo/Central credentials are host-supplied through getAuthToken() and stored in storage.kv (encrypted at rest per 09 · Offline storage); the transport never logs URLs that embed a token-in-path (/collector/{token}/…, /v1/key/{token}/… are redacted in SyncLogEntry). Digest auth is implemented only for the generic profile and always over HTTPS (policy.requireHttps). Server-side bridges (§5.2) use service tokens that live on the server, never in a device bundle. Webhook receivers verify X-Rasd-Signature in constant time; the sender refuses private-network and link-local destinations (SSRF), follows no redirects, and pins a 10 s timeout.
PII and data protection. Exports honour bind.sensitive: redact=drop|hash (§6), the data dictionary marks sensitive columns so analysts can build views without them, and submission.completed webhooks carry sensitive fields only when the webhook is created with includeSensitive: true (default false, logged). RapidPro/DHIS2 bridges are documented to map non-sensitive fields only. Signed attachment URLs expire in 1 h and are bound to the org. Every export and unredacted download is written to the server audit log with actor, filter and row count. Imported public_key settings do not by themselves encrypt anything (§2.1) — the report says so, so an author does not assume ODK-style encryption is active on Rasd devices. Nothing in this document requires a valid license: storage.export(), importXlsform(), exportXlsform() and the CLI work in every license state (spine §9, P6).
Performance. Import target: 1,000 survey rows ≤ 3 s in Node, ≤ 6 s in a low-end Android WebView worker (NFR-008); memory ≤ 3× workbook size; the XPath → REL rewriter is single-pass over a token stream. Export jobs stream (CSV via back-pressured writer, XLSX via the xlsx streaming API, zip via streaming deflate) so a 1 M-row export never materialises in RAM; jobs are resumable per table. flattenSubmission() is O(elements) per row and shares the header plan across a job. OData $skiptoken pages default to 1,000 rows. The openrosa transport sorts media by size and fills chunks greedily so the number of POSTs is minimal for the ACL; serialisation of a 500-instance repeat stays under 50 ms on the reference device.
Accessibility and i18n. Exports and dictionaries carry labels in the requested locale with fallback to settings.defaultLocale; XLSX sheets set RTL direction for RTL locales and freeze the header row; CSV is UTF-8 with BOM so Excel opens Arabic correctly; column names never contain non-ASCII (they are element names), so downstream tools with ASCII-only identifiers still work. Import warnings shown in the builder are announced through the builder's live region and are keyboard-navigable to the offending cell coordinates (08 · Builder, 13 · i18n, RTL & accessibility); the conformance dashboard page in the docs meets the same WCAG 2.2 AA bar as the rest of the site.
14. Acceptance criteria
-
importXlsform()handles every column in §2 with the documentedextplacement; unknown columns/sheets survive import → export unchanged (property-based test); everyW_*/I_*code has a fixture and a snapshot message with sheet/row/column. - Corpus job: ≥ 90 % zero-error imports, ≥ 95 % expressions parsed; unchanged import → export of the pyxform fixtures is canonical-XML-identical outside the allow-list;
target: 'kobo'refoldsbegin_score/begin_rank/begin_kobomatrix. - Every Rasd-only element type has an export downgrade (
W_EXPORT_DOWNGRADED) and an import path that restores it fromext. -
serializeXformInstance()output validates against pyxform-compiled XForms (space-separated multi-selects,lat lng alt acc,;-joined traces, offset datetimes,uuid:instanceID, non-relevant nodes omitted);parseXformInstance(serializeXformInstance(x)) ≡ xfor all value types. -
openrosatransport passes contract tests for all four profiles including 202-duplicate, 409-conflict and 413-rechunk; the nightly live test against dockerised Central and kobo-install is green. - Exports: a 3-version form yields union columns with per-row
__version; Kobo- and Central-mode CSV open in Excel/Power BI with the documented headers; GeoJSON validates (RFC 7946); media zip sha256 matches; XLSX splits above 1,048,576 rows; the OData feed works with an unchanged Central Power BI template. - Webhook signatures verify with the published snippet; replays carry the same
X-Rasd-Delivery; retries stop after 10 attempts and appear in the redelivery UI. -
rasd dictionarylists every value-bearing element, choice list and dataset in CSV/XLSX/MD/JSON;catalog.jsonvalidates against the Dublin Core term whitelist;importSurveyJs()imports the SurveyJS documentation samples with onlyW_SURVEYJS_*warnings. - Every failure mode in §12 has a test: XXE/DTD payloads and a 100 MB workbook are rejected with the documented errors and bounded memory; import of a fuzzer corpus (10k mutated workbooks) never crashes, hangs past the timeout, or produces an RFD that fails
validateFormDefinition. - Security: token-in-path URLs never appear in logs (
SyncLogEntrysnapshot test);redact=drop|hashremoves/obscures everybind.sensitivecolumn including inside repeat tables and GeoJSON properties; CSV formula-injection guard covers text cells in all export formats. - Performance: 1,000-row import ≤ 3 s Node / ≤ 6 s WebView worker; 1 M-row CSV export completes streaming with < 512 MiB server RSS; import of the largest corpus form leaves the builder main thread responsive (worker-based).
Open questions
- Should imported XLSForm groups default to
nestData: true(exact Kobo/ODK data shape) rather than transparent + recorded path? Transparent keeps REL simple; nested keeps exports and bridges trivially identical. - Which humanitarian XLSForm sets (REACH/IOM/UNHCR/WFP) may be redistributed in the public corpus, and under which terms?
- Is the
@rasd/xlsform/rel-odkcompat pack (boolean-from-string,checklist,digest, …) worth maintaining, or should unmapped functions stay warnings until demand appears? - Do we implement the ODK RSA-2048/AES-256-CFB encryption envelope in the
openrosatransport for Central managed encryption in phase 3, or defer (Kobo cannot decrypt anyway)? - Should
submission.completedwebhooks optionally carry the flattened Kobo-style row (flattenSubmission) to spare receivers a second call? - XLSForm
trigger(xforms-value-changed) needs an "on change of${src}" trigger; proposeTrigger.on: { change: "${src}" }in RFD 1.1 so the importer stops approximating. quickauto-advance: promoteprops.ext["org.getodk.xlsform"].autoAdvanceto a first-classselect_oneprops.autoAdvancein RFD 1.1 (§2.6)?- Catalog metadata: promote
meta.ext["dev.rasd.catalog"]to a first-classmeta.catalogin RFD 1.1, and emit JSON-LD/DCAT for HDX?
Related documents
- 00 · Design spine · 02 · Requirements (FR-130–135) · 03 · Architecture (§8
SyncTransport) · 04 · Form schema spec (§12ext, §19 concept mapping) · 05 · Logic & expressions (§16 REL ↔ XPath ↔ SurveyJS) · 08 · Builder · 09 · Offline storage · 10 · Sync protocol (§9.5 webhooks/exports, §10 OpenRosa) · 13 · i18n, RTL & accessibility · 16 · Security & data protection · 17 · API reference (§12@rasd/xlsform, §13 CLI) · 18 · Engineering practices · 19 · Roadmap (E2.5, E3.5) · 21 · Getting started - Research: 01 · UN field data-collection landscape · 02 · React form-builder libraries · 10 · Pricing, GTM & procurement · 12 · Form versioning · 14 · ODK/Kobo/OpenRosa interop & XLSForm fidelity