validation docs
This commit is contained in:
2
canvas
2
canvas
Submodule canvas updated: e76cda8e0c...a4f0216878
2
server
2
server
Submodule server updated: 93ceaf4267...99bc97f771
@@ -1,82 +0,0 @@
|
||||
# Validation Architecture
|
||||
|
||||
Validation is split into three ownership layers.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Server[server<br/>stores simple settings<br/>binds table fields<br/>enforces writes]
|
||||
Core[validation-core<br/>owns meaning<br/>resolves sets<br/>runs pure validation]
|
||||
Canvas[canvas<br/>editor integration<br/>masking while typing<br/>UI feedback]
|
||||
Common[common/proto<br/>wire format]
|
||||
|
||||
Server --> Core
|
||||
Canvas --> Core
|
||||
Server --> Common
|
||||
Canvas --> Common
|
||||
```
|
||||
|
||||
## Rule
|
||||
|
||||
`server` stores dumb, serializable settings. `validation-core` owns what those
|
||||
settings mean. `canvas` uses the resolved result for editing behavior.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Settings[ValidationSettings<br/>serializable data]
|
||||
Rule[ValidationRule<br/>named reusable fragment]
|
||||
Set[ValidationSet<br/>ordered rules]
|
||||
Config[ValidationConfig<br/>resolved runtime config]
|
||||
Result[ValidationResult]
|
||||
|
||||
Rule --> Settings
|
||||
Set --> Rule
|
||||
Set --> Settings
|
||||
Settings --> Config
|
||||
Config --> Result
|
||||
```
|
||||
|
||||
## Current Data Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant DB as server DB
|
||||
participant Server as server
|
||||
participant Core as validation-core
|
||||
participant Client as client/canvas
|
||||
|
||||
DB->>Server: stored field validation settings
|
||||
Server->>Core: interpret shared validation primitives
|
||||
Server->>Client: gRPC validation config
|
||||
Client->>Core: resolve/use shared primitives
|
||||
Client->>Client: canvas editing, masks, errors
|
||||
```
|
||||
|
||||
## Set Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
RuleA[digits-only rule]
|
||||
RuleB[phone-length rule]
|
||||
RuleC[phone-mask rule]
|
||||
Set[phone set]
|
||||
Assignment[column assignment]
|
||||
Stored[server stored settings<br/>set name + resolved config]
|
||||
Runtime[server/canvas runtime]
|
||||
|
||||
RuleA --> Set
|
||||
RuleB --> Set
|
||||
RuleC --> Set
|
||||
Set --> Assignment
|
||||
Assignment --> Stored
|
||||
Stored --> Runtime
|
||||
```
|
||||
|
||||
The server stores reusable rules and sets, and field application stores a
|
||||
resolved snapshot:
|
||||
|
||||
```text
|
||||
field customer_phone uses set phone
|
||||
resolved settings = {...}
|
||||
```
|
||||
|
||||
That keeps backend enforcement stable even if the reusable set changes later.
|
||||
493
validation-core/docs/validation_architecture.md
Normal file
493
validation-core/docs/validation_architecture.md
Normal file
@@ -0,0 +1,493 @@
|
||||
# Validation
|
||||
|
||||
This document is the frontend guide for the validation system.
|
||||
|
||||
The important idea: reusable validation is built from **rules** and **sets**.
|
||||
The frontend creates and manages those. When a set is applied to a table field,
|
||||
the server resolves it into the existing `FieldValidation` shape, and the form
|
||||
runtime continues to work through the normal table-validation flow.
|
||||
|
||||
## Ownership
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Core[validation-core<br/>validation meaning<br/>rule/set merge rules]
|
||||
Server[server<br/>stores rules/sets<br/>applies sets<br/>enforces writes]
|
||||
Common[common/proto<br/>gRPC contract]
|
||||
Client[client/frontend<br/>rule/set UI<br/>calls gRPC]
|
||||
Canvas[canvas<br/>field editing<br/>mask display<br/>local feedback]
|
||||
|
||||
Server --> Core
|
||||
Canvas --> Core
|
||||
Client --> Common
|
||||
Server --> Common
|
||||
Client --> Canvas
|
||||
```
|
||||
|
||||
`server` stores simple serializable settings. `validation-core` owns how those
|
||||
settings combine. `canvas` uses resolved field validation to guide editing.
|
||||
|
||||
## Terms
|
||||
|
||||
| Term | Meaning |
|
||||
| --- | --- |
|
||||
| `FieldValidation` | Existing per-column validation config from `common/proto/table_validation.proto`. This is what forms/canvas already consume. |
|
||||
| `ValidationRule` | One named reusable fragment, for example `digits-only`, `phone-length`, or `required`. Stored by the server as a `FieldValidation` fragment with no meaningful `dataKey`. |
|
||||
| `ValidationSet` | Ordered collection of rule names, for example `phone = [required, phone-length, digits-only, phone-mask]`. |
|
||||
| Applied validation | A resolved snapshot of a set written to `table_validation_rules` for a concrete `(table, dataKey)`. |
|
||||
| Snapshot | Applying a set copies the resolved config to a field. Later edits to the set do not automatically update fields that were already applied. |
|
||||
|
||||
## What Backend Enforces
|
||||
|
||||
Backend write validation enforces only server-relevant parts:
|
||||
|
||||
| FieldValidation part | Backend | Canvas/frontend |
|
||||
| --- | --- | --- |
|
||||
| `required` | Yes | Yes |
|
||||
| `limits` | Yes | Yes |
|
||||
| `pattern` | Yes | Yes |
|
||||
| `allowed_values` | Yes | Yes |
|
||||
| `mask` | Partly: raw value length/literals | Yes: display/editing mask |
|
||||
| `formatter` | No | Yes |
|
||||
| `external_validation_enabled` | No | Yes/UI hint |
|
||||
|
||||
`mask` is visual metadata, but the backend still uses it to reject incorrectly
|
||||
submitted raw values. Example: if the mask is `(###) ###-####`, the backend
|
||||
expects the stored value to be raw digits, not `(123) 456-7890`.
|
||||
|
||||
## Main User Flow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant UI as Frontend UI
|
||||
participant API as TableValidationService
|
||||
participant DB as Server DB
|
||||
participant Form as Existing Form Runtime
|
||||
|
||||
UI->>API: UpsertValidationRule(required)
|
||||
UI->>API: UpsertValidationRule(digits-only)
|
||||
UI->>API: UpsertValidationRule(phone-length)
|
||||
UI->>API: UpsertValidationSet(phone: [required, phone-length, digits-only])
|
||||
UI->>API: ApplyValidationSet(profile, table, dataKey, phone)
|
||||
API->>DB: write resolved FieldValidation snapshot
|
||||
Form->>API: GetTableValidation(profile, table)
|
||||
API->>Form: resolved FieldValidation for dataKey
|
||||
```
|
||||
|
||||
After `ApplyValidationSet`, the existing form code does not need to know that a
|
||||
set was used. It receives normal `FieldValidation`.
|
||||
|
||||
## API
|
||||
|
||||
All APIs live on `TableValidationService`.
|
||||
|
||||
### Rules
|
||||
|
||||
Create or update one reusable rule:
|
||||
|
||||
```text
|
||||
UpsertValidationRule(UpsertValidationRuleRequest)
|
||||
```
|
||||
|
||||
Request shape:
|
||||
|
||||
```text
|
||||
profileName: string
|
||||
rule:
|
||||
name: string
|
||||
description: optional string
|
||||
validation: FieldValidation
|
||||
```
|
||||
|
||||
Frontend rules:
|
||||
|
||||
- `rule.name` is required and unique inside a profile.
|
||||
- `rule.validation.dataKey` is ignored by the server.
|
||||
- A rule should usually configure one logical fragment.
|
||||
- Examples: `required`, `phone-length`, `digits-only`, `phone-mask`.
|
||||
|
||||
List rules:
|
||||
|
||||
```text
|
||||
ListValidationRules({ profileName })
|
||||
```
|
||||
|
||||
Delete rule:
|
||||
|
||||
```text
|
||||
DeleteValidationRule({ profileName, name })
|
||||
```
|
||||
|
||||
Deleting a rule removes it from future reusable composition. Already applied
|
||||
field snapshots are not changed.
|
||||
|
||||
### Sets
|
||||
|
||||
Create or update one reusable set:
|
||||
|
||||
```text
|
||||
UpsertValidationSet(UpsertValidationSetRequest)
|
||||
```
|
||||
|
||||
Request shape:
|
||||
|
||||
```text
|
||||
profileName: string
|
||||
set:
|
||||
name: string
|
||||
description: optional string
|
||||
ruleNames: repeated string
|
||||
```
|
||||
|
||||
Frontend rules:
|
||||
|
||||
- `set.name` is required and unique inside a profile.
|
||||
- `ruleNames` must contain at least one rule.
|
||||
- `ruleNames` are ordered.
|
||||
- Every rule name must already exist.
|
||||
- Duplicate rule names in the same set are rejected.
|
||||
- Conflicting singleton fragments are rejected.
|
||||
|
||||
Singleton fragments are:
|
||||
|
||||
```text
|
||||
limits
|
||||
allowed_values
|
||||
mask
|
||||
formatter
|
||||
```
|
||||
|
||||
That means a set cannot currently contain two rules that both define `limits`.
|
||||
Pattern rules are additive: multiple rules with `pattern` are merged into one
|
||||
combined pattern.
|
||||
|
||||
List sets:
|
||||
|
||||
```text
|
||||
ListValidationSets({ profileName })
|
||||
```
|
||||
|
||||
Response includes each set plus `resolvedValidation`, so the frontend can show
|
||||
what the set expands to.
|
||||
|
||||
Delete set:
|
||||
|
||||
```text
|
||||
DeleteValidationSet({ profileName, name })
|
||||
```
|
||||
|
||||
Deleting a set does not change already applied fields.
|
||||
|
||||
### Apply Set To Field
|
||||
|
||||
Apply a reusable set to one field:
|
||||
|
||||
```text
|
||||
ApplyValidationSet(ApplyValidationSetRequest)
|
||||
```
|
||||
|
||||
Request shape:
|
||||
|
||||
```text
|
||||
profileName: string
|
||||
tableName: string
|
||||
dataKey: string
|
||||
setName: string
|
||||
```
|
||||
|
||||
Server behavior:
|
||||
|
||||
1. Loads the set.
|
||||
2. Loads its ordered rules.
|
||||
3. Resolves/merges them through `validation-core`.
|
||||
4. Validates that `dataKey` exists in the table definition.
|
||||
5. Writes the resolved config into existing `table_validation_rules`.
|
||||
|
||||
This is a snapshot. If the user later edits the `phone` set, fields that already
|
||||
used `phone` keep their old resolved config until the set is applied again.
|
||||
|
||||
## FieldValidation Guide
|
||||
|
||||
Rules and direct field validation both use `FieldValidation`.
|
||||
|
||||
### Required
|
||||
|
||||
```text
|
||||
required: true
|
||||
```
|
||||
|
||||
Backend rejects missing or empty values.
|
||||
|
||||
### Limits
|
||||
|
||||
```text
|
||||
limits:
|
||||
min: 10
|
||||
max: 10
|
||||
warnAt: optional
|
||||
countMode: CHARS | BYTES | DISPLAY_WIDTH
|
||||
```
|
||||
|
||||
Backend enforces `min` and `max`. `warnAt` is mainly UI feedback.
|
||||
|
||||
### Pattern
|
||||
|
||||
Pattern rules validate characters at positions.
|
||||
|
||||
Example digits-only:
|
||||
|
||||
```text
|
||||
pattern:
|
||||
rules:
|
||||
- position:
|
||||
kind: PATTERN_POSITION_FROM
|
||||
start: 0
|
||||
constraint:
|
||||
kind: CHARACTER_CONSTRAINT_NUMERIC
|
||||
```
|
||||
|
||||
Useful constraints:
|
||||
|
||||
```text
|
||||
CHARACTER_CONSTRAINT_ALPHABETIC
|
||||
CHARACTER_CONSTRAINT_NUMERIC
|
||||
CHARACTER_CONSTRAINT_ALPHANUMERIC
|
||||
CHARACTER_CONSTRAINT_EXACT
|
||||
CHARACTER_CONSTRAINT_ONE_OF
|
||||
CHARACTER_CONSTRAINT_REGEX
|
||||
```
|
||||
|
||||
Pattern fragments from multiple rules are merged.
|
||||
|
||||
### Allowed Values
|
||||
|
||||
```text
|
||||
allowed_values:
|
||||
values: ["open", "closed"]
|
||||
allow_empty: false
|
||||
case_insensitive: true
|
||||
```
|
||||
|
||||
Backend rejects values not in the list.
|
||||
|
||||
### Mask
|
||||
|
||||
```text
|
||||
mask:
|
||||
pattern: "(###) ###-####"
|
||||
input_char: "#"
|
||||
template_char: "_"
|
||||
```
|
||||
|
||||
Canvas uses this for display/editing. Backend expects raw values without mask
|
||||
literals.
|
||||
|
||||
### Formatter
|
||||
|
||||
```text
|
||||
formatter:
|
||||
type: "PhoneFormatter"
|
||||
options: []
|
||||
description: optional
|
||||
```
|
||||
|
||||
Formatter is resolved client-side. Backend stores it but does not execute it.
|
||||
|
||||
### External Validation
|
||||
|
||||
```text
|
||||
external_validation_enabled: true
|
||||
```
|
||||
|
||||
This is a frontend/UI hint. Backend stores it but does not perform external
|
||||
validation.
|
||||
|
||||
## Recommended Frontend Screens
|
||||
|
||||
### Rule List
|
||||
|
||||
Show all rules for a profile.
|
||||
|
||||
Actions:
|
||||
|
||||
```text
|
||||
create rule
|
||||
edit rule
|
||||
delete rule
|
||||
preview rule config
|
||||
```
|
||||
|
||||
### Rule Editor
|
||||
|
||||
Build a `ValidationRuleDefinition`.
|
||||
|
||||
Recommended UI:
|
||||
|
||||
```text
|
||||
name
|
||||
description
|
||||
required toggle
|
||||
limits section
|
||||
pattern section
|
||||
allowed values section
|
||||
mask section
|
||||
formatter section
|
||||
external validation toggle
|
||||
```
|
||||
|
||||
For v1, encourage one fragment per rule. Example: create `phone-length` and
|
||||
`digits-only` separately, instead of one huge rule.
|
||||
|
||||
### Set List
|
||||
|
||||
Show all sets for a profile.
|
||||
|
||||
Use `ListValidationSets`, because it returns `resolvedValidation`.
|
||||
|
||||
Actions:
|
||||
|
||||
```text
|
||||
create set
|
||||
edit set
|
||||
delete set
|
||||
preview resolved validation
|
||||
```
|
||||
|
||||
### Set Editor
|
||||
|
||||
Build a `ValidationSetDefinition`.
|
||||
|
||||
Recommended UI:
|
||||
|
||||
```text
|
||||
name
|
||||
description
|
||||
ordered rule picker
|
||||
resolved preview
|
||||
```
|
||||
|
||||
When rule ordering changes, call `UpsertValidationSet` and then refresh
|
||||
`ListValidationSets`.
|
||||
|
||||
### Apply Set
|
||||
|
||||
On the table/field validation screen, add:
|
||||
|
||||
```text
|
||||
Apply validation set
|
||||
```
|
||||
|
||||
Flow:
|
||||
|
||||
1. Load sets with `ListValidationSets`.
|
||||
2. User selects a set.
|
||||
3. Call `ApplyValidationSet(profileName, tableName, dataKey, setName)`.
|
||||
4. Refresh `GetTableValidation(profileName, tableName)`.
|
||||
|
||||
The field should now behave exactly like a directly configured field validation.
|
||||
|
||||
## Example: Phone
|
||||
|
||||
Create rule `required`:
|
||||
|
||||
```text
|
||||
validation:
|
||||
required: true
|
||||
```
|
||||
|
||||
Create rule `phone-length`:
|
||||
|
||||
```text
|
||||
validation:
|
||||
limits:
|
||||
min: 10
|
||||
max: 10
|
||||
countMode: CHARS
|
||||
```
|
||||
|
||||
Create rule `digits-only`:
|
||||
|
||||
```text
|
||||
validation:
|
||||
pattern:
|
||||
rules:
|
||||
- position:
|
||||
kind: PATTERN_POSITION_FROM
|
||||
start: 0
|
||||
constraint:
|
||||
kind: CHARACTER_CONSTRAINT_NUMERIC
|
||||
```
|
||||
|
||||
Create rule `phone-mask`:
|
||||
|
||||
```text
|
||||
validation:
|
||||
mask:
|
||||
pattern: "(###) ###-####"
|
||||
input_char: "#"
|
||||
```
|
||||
|
||||
Create set `phone`:
|
||||
|
||||
```text
|
||||
ruleNames:
|
||||
- required
|
||||
- phone-length
|
||||
- digits-only
|
||||
- phone-mask
|
||||
```
|
||||
|
||||
Apply set:
|
||||
|
||||
```text
|
||||
profileName: "default"
|
||||
tableName: "customers"
|
||||
dataKey: "customer_phone"
|
||||
setName: "phone"
|
||||
```
|
||||
|
||||
Then refresh:
|
||||
|
||||
```text
|
||||
GetTableValidation(default, customers)
|
||||
```
|
||||
|
||||
The response contains a normal `FieldValidation` for `customer_phone`.
|
||||
|
||||
## Important UX Notes
|
||||
|
||||
- Applying a set is not a live link.
|
||||
- Editing a rule or set does not mutate fields where it was already applied.
|
||||
- To update a field after set changes, apply the set again.
|
||||
- If a set has conflicting singleton rules, the server rejects it.
|
||||
- For now, the system does not store field metadata like `sourceSetName` on
|
||||
applied fields. The field only stores the resolved validation snapshot.
|
||||
|
||||
## Files
|
||||
|
||||
Core model:
|
||||
|
||||
```text
|
||||
validation-core/src/set.rs
|
||||
validation-core/src/config.rs
|
||||
```
|
||||
|
||||
Wire contract:
|
||||
|
||||
```text
|
||||
common/proto/table_validation.proto
|
||||
```
|
||||
|
||||
Server implementation:
|
||||
|
||||
```text
|
||||
server/src/table_validation/get/service.rs
|
||||
server/src/table_validation/post/repo.rs
|
||||
server/src/table_validation/config.rs
|
||||
```
|
||||
|
||||
Storage:
|
||||
|
||||
```text
|
||||
server/migrations/20260506170000_create_validation_rules_and_sets.sql
|
||||
```
|
||||
Reference in New Issue
Block a user