-Content-Type: application/json
+The value is stored as a SHA-256 hash; a value that is already 64 characters long is treated as hashed and stored unchanged. API responses never return a hashed field: reads, queries and commit responses replace its value with `**REDACTED**`, and writing the placeholder back keeps the stored hash. Seeing `**REDACTED**` means the field is hashed and protected, not that the password was exposed. To check a password, sign in with it rather than reading the field.
-{ "Name": "New Hire", "Department": "Engineering", "CreatedBy": "bob@acme.com" }
+And one more example where one field gets trimmed and converted to lowercase while the second field is converted to uppercase:
-// Response (200 OK): new entry
+{% raw %}
+```json
+[
+ {
+ "type": "operations",
+ "runOn": ["beforeSave"],
+ "payload": {
+ "Email": ["lower", "trim"],
+ "Name": ["upper"]
+ }
+ }
+]
```
+{% endraw %}
-**Queries that fail:**
-
-```js
-// Alice (Manager) tries to read Marketing records — fails
-// (the require condition demands Department matching Alice's own department "Engineering";
-// "Marketing" does not match, so the first rule is skipped; no other rule grants broad access)
-await connection.find({
- where: { Department: 'Marketing' }
-});
-
-// Bob tries to read all records without filtering by email — fails
-// (the second rule requires Email matching Bob's session; without a where clause,
-// the require condition is not met, the rule is skipped, and no subsequent rule grants access)
-await connection.find();
-
-// Bob tries to read Alice's record — fails
-// (Email does not match Bob's session)
-await connection.find({
- where: { Email: 'alice@acme.com' }
-});
-
-// Bob tries to insert a record in Marketing — fails
-// (Department does not match Bob's department "Engineering")
-await connection.insert({
- Name: 'Spy',
- Department: 'Marketing',
- CreatedBy: 'bob@acme.com'
-});
+This last example shows string interpolation being compiled with Handlebars:
-// Bob tries to insert with a different CreatedBy — fails
-// (CreatedBy must equal Bob's email)
-await connection.insert({
- Name: 'Fake',
- Department: 'Engineering',
- CreatedBy: 'alice@acme.com'
-});
+{% raw %}
+```json
+[
+ {
+ "type": "operations",
+ "runOn": ["beforeSave"],
+ "payload": {
+ "Content": "Foo bar",
+ "AnotherContent": "Foo {{ email }}"
+ }
+ }
+]
```
+{% endraw %}
-## Data requirements and query validation
-
-The `require` property defines conditions that incoming queries must satisfy. This is not the same as querying data — it controls how queries are **assessed against the rule's data requirements**.
-
-Components such as List from Data Source , Chart and Form may issue broad queries that do not satisfy your read requirements. Configure their filters or use custom queries that satisfy the intended restrictions. Adding a permissive fallback rule grants broader access even when a stricter rule appears first; an appId restriction scopes a rule to apps, not to individual components.
-
-The behavior of `require` varies by operation type:
-
-- **For `select` queries:** Requirements validate the submitted filter. For example, if a rule requires {% raw %}`{ "Email": { "equals": "{{user.[Email]}}" } }`{% endraw %}, a `find()` call must include `where: { Email: user.Email }` (or an equivalent supported filter) for that rule to grant access. Direct reads by entry ID instead validate the stored entry data. To obtain session data client-side, use the [Session API](API/fliplet-session.html).
-- **For `delete` operations:** Direct deletion by entry ID validates the stored entry data and does not require a client `where` filter. Query-based deletion requires a nonempty filter and read access, then validates delete access against each matched row's stored data. Explicit commit deletion IDs provide an array of existing row data; other bulk deletion paths, such as replacing imported data, can omit row data. Do not assume every delete evaluates a submitted filter or always provides stored rows.
-- **For `insert` operations:** Required fields and value conditions validate the submitted data.
-- **For `update` operations:** String requirements check that the submitted data includes the named fields. Object-form value conditions check the existing stored row, including stored rows loaded for commit updates. They do not restrict proposed replacement values. The current custom-rule `query` context also cannot reliably enforce allowed new values or an exact set of write fields. Use a server write mechanism that independently enforces such constraints, or leave the constrained write unavailable.
-
-Rules grant access independently: If any require condition fails, that rule does not grant access and evaluation continues for reads and writes alike. Access is denied only if no applicable rule grants it. Remove or narrow any other rule that would admit requests your restrictions are intended to reject; ordering a strict rule first does not make a broader fallback safe.
-
-### Data requirement types
+---
-`require` accepts an array of strings (required field names) and objects (field conditions). Four requirement types are available:
+# Securing Fliplet data sources
+URL: https://developers.fliplet.com/Data-source-security.md
-| Requirement type | Description | Query patterns that satisfy this requirement |
-|---|---|---|
-| Field is required | The query must include the specified column | Any query that provides the column as a key |
-| Field equals | The query value for the column must match exactly | `{ Field: value }` or `{ Field: { $eq: value } }` |
-| Field not equals | The query value for the column must not match the specified value | `{ Field: { $ne: value } }` or any query where the field's value differs from the specified value |
-| Field contains | The query value for the column must contain the specified substring | `{ Field: value }` or `{ Field: { $iLike: value } }` or `{ Field: { $like: value } }` |
+# Securing Fliplet data sources
-These requirement types (equals, notequals, contains) are not query operators — they define how the security rule validates incoming queries. For the full list of query operators you can use when reading and writing data, see the query operators reference .
+Protect app data with server-evaluated access rules, then test the allowed and denied operations as real app users. Rules control access; browser checks and successful Studio administrative requests do not prove authorization.
-### Handlebars templating
+## Configure a policy
-Condition values can reference the logged-in user's session data using Handlebars syntax. This enables dynamic, per-user access scoping. Use bracket notation (e.g., `[Email]`) for field names — it is required for names with spaces or special characters (e.g., {% raw %}`{{user.[First Name]}}`{% endraw %}), and also works for simple names. Plain dot notation ({% raw %}`{{user.Email}}`{% endraw %}) works only for field names without spaces.
+1. Open **App Data** in Fliplet Studio, select the data source and open **Access Rules**.
+2. Define the actors, operations and fields that need protection. Start with an explicit empty rules array (`[]`) for a source that should deny app access until configured. Missing or null rules retain legacy unrestricted access.
+3. Choose a [security example with explicit scope](API/datasources/security-examples.md) and adapt its data shape and login-source binding. Save the rules, then read back the configuration.
+4. Test both allowed and denied operations with [Enforce security enabled in preview](API/datasources/testing-security.md). Confirm persisted results using a fresh read.
-For per-account ownership, use an owner key that is unique to each account, stable and protected from end-user changes through every write path. This requirement applies to the owner key; shared fields such as `Role` and `Department` do not need to be unique. Before matching an account to private records already assigned to an email address or external identity, verify control of that identity or provision the account through a trusted process. A self-chosen email at public signup does not establish entitlement to those records. Public signup can support users creating their own records when a stable account key and server-enforced rules bind those records to the authenticated account.
+A minimal public read-only policy is:
-- {% raw %}`{{user.[Email]}}`{% endraw %} — the user's email address
-- {% raw %}`{{user.[Department]}}`{% endraw %} — the user's department
-- {% raw %}`{{user.[Role]}}`{% endraw %} — the user's role
-- {% raw %}`{{user.[ID]}}`{% endraw %} — the user's data source entry ID
+```json
+[
+ { "type": ["select"], "allow": "all" }
+]
+```
-### Require syntax
+Use this only for data intended for anonymous readers. It grants no insert, update or delete operation. The [public lookup example](API/datasources/security-examples.md#public-lookup) pairs this rule with sample rows and SDK calls.
-**Required fields** (string) — the query must include these columns:
+## Choose the right guide
-```json
-"require": ["Email", "FirstName", "Department"]
-```
+| Task | Guide |
+|---|---|
+| Understand rule order, identity, requirements and endpoint differences | [Security rule reference](API/datasources/security-rules.md) |
+| Explore public, shared, owner-filtered reads, profile or role/state examples | [Security examples](API/datasources/security-examples.md) |
+| Prove allowed and denied app behavior | [Authenticated preview testing](API/datasources/testing-security.md) |
+| Connect, query or write through JavaScript | [Fliplet.DataSources reference](API/fliplet-datasources.md) |
+| Configure authentication or hooks | [App authentication](API/v3/auth.md), [Session API](API/fliplet-session.md) and [Data source hooks](Data-Source-Hooks.md) |
-**Field conditions** (object with operator) — the query must match these conditions:
+## Check the policy's limits
-{% raw %}
-```json
-"require": [
- { "Email": { "equals": "{{user.[Email]}}" } },
- { "Status": { "notequals": "Archived" } },
- { "Department": { "contains": "{{user.[Department]}}" } }
-]
-```
-{% endraw %}
+Rules grant access in order. A failed rule falls through for writes as well as reads, so a later broad grant can defeat an earlier restriction.
-You can mix both formats in the same array:
+Object conditions on updates check the **stored row**, while string requirements check submitted key presence. A condition on stored `Owner` or `Status` decides whether the caller may update that row; it does not stop them changing those fields. Exclude the fields each actor must not set from that actor's insert and update rules: single-record writes and `commit()` both drop excluded fields, and commit updates merge onto the stored row. Bulk `append()` and `replaceWith()` do not apply exclusions. Custom scripts receive a copied input; assigning audit fields to that copy does not reliably persist them.
-{% raw %}
-```json
-"require": [
- "Title",
- "Description",
- { "CreatedBy": { "equals": "{{user.[Email]}}" } },
- { "Status": { "equals": "Draft" } }
-]
-```
-{% endraw %}
+For workflows where one person submits and another decides, see [requests and approvals](API/datasources/security-examples.md#requests-and-approvals).
-For full examples of `require` in practice — including sample data, succeed queries, and fail queries — see [role-based access with protected fields](#example-role-based-access-with-protected-fields) and [department-scoped access](#example-department-scoped-access) above.
+## Existing section links
-## Custom security rules
+These destinations retain links to the previous sections of this guide.
-For advanced logic beyond what the standard rule properties support, you can write custom JavaScript security rules.
+Important: When a rule has a script, the script is the sole determinant of access. Standard rule fields like allow and type are ignored — the script runs regardless of login status or operation type. Your script must perform its own identity and operation checks (e.g., if (!user) return { granted: false };). If the script does not return a value (e.g., an unhandled operation type falls through without a return), access is denied by default .
+[Access rule structure](API/datasources/security-rules.md#rule-properties)
-When writing a custom rule, these variables are available in the script context:
+Important: You must return an object — bare boolean values (e.g., return true) are not supported and will be treated as a denial. Always use return { granted: true } or return { granted: false }.
+[Data requirements and query validation](API/datasources/security-rules.md#requirements-by-operation)
-Cross-data-source lookups add a database round-trip per request and are not cached. Keep queries efficient and use findOne when you only need a single record. Script execution has a 3-second timeout — if the script does not return within 3 seconds, access is denied. Async operations (like DataSources queries) are supported via await.
+[Custom security rules](API/datasources/security-rules.md#custom-scripts)
-Connect using the data source ID (number) or name (string):
+Use DataSources('Name') (data source name) instead of DataSources(123) (ID) in custom scripts. Data source names are preserved during app clone, so name-based lookups continue to work without manual remapping.
+[Reading data from other data sources](API/datasources/security-rules.md#reading-another-data-source)
---
@@ -43241,7 +42731,7 @@ Every Fliplet JS API available to V3 apps, grouped by capability category. Each
## Data
- [`Fliplet.Cache`](https://developers.fliplet.com/API/core/cache.html) **(preloaded)** — Run async operations once and memoize their results, with optional expiry and background refresh.
-- [`Fliplet.DataSources`](https://developers.fliplet.com/API/fliplet-datasources.html) — Connect to, query, insert, update, and delete records in Fliplet Data Sources from inside an app. All methods are promise-based.
+- [`Fliplet.DataSources`](https://developers.fliplet.com/API/fliplet-datasources.html) — Connect to Fliplet data sources and find references for queries, writes, subscriptions, native storage and source management.
- [`Fliplet.DataSources.Encryption`](https://developers.fliplet.com/API/fliplet-encryption.html) — Automatically encrypt and decrypt selected Data Source columns on-device by registering a private key and column list.
- [`Fliplet.Storage and Fliplet.App.Storage`](https://developers.fliplet.com/API/core/storage.html) **(preloaded)** — Persist JSON-serializable values to device or browser storage, scoped globally or to the current app.
@@ -43256,7 +42746,7 @@ Every Fliplet JS API available to V3 apps, grouped by capability category. Each
## Communications
-- [`Fliplet.Chat`](https://developers.fliplet.com/API/fliplet-chat.html) — Build one-to-one, group, and public-channel chat features. Fliplet.Chat owns the conversations and messages data sources internally — supply only the contacts list (who can chat with whom).
+- [`Fliplet.Chat`](https://developers.fliplet.com/API/fliplet-chat.html) — Chat API for V2 apps with a Chat component. Not available to V3 apps: build messaging on the app's own data sources.
- [`Fliplet.Communicate`](https://developers.fliplet.com/API/fliplet-communicate.html) — Send email, SMS, push notifications, and share URLs from a Fliplet app using a single Communicate namespace.
- [`Fliplet.Navigator.Notifications`](https://developers.fliplet.com/API/core/notifications.html) **(preloaded)** — Check notification support, request permission, and send local device notifications from JavaScript.
- [`Fliplet.Notifications`](https://developers.fliplet.com/API/fliplet-notifications.html) — Read, send, and schedule in-app and push notifications in Fliplet apps, with support for scopes, read receipts, and badge counts.
diff --git a/docs/.well-known/llms-v3-libraries.json b/docs/.well-known/llms-v3-libraries.json
index 54a82eb0..8c92de6b 100644
--- a/docs/.well-known/llms-v3-libraries.json
+++ b/docs/.well-known/llms-v3-libraries.json
@@ -1,6 +1,6 @@
{
"version": 1,
- "generatedAt": "2026-10-16T11:33:39.654Z",
+ "generatedAt": "2026-10-22T12:21:52.485Z",
"libraries": [
{
"package": "fliplet-analytics-spa",
@@ -84,7 +84,7 @@
"package": "fliplet-chat",
"namespace": "Fliplet.Chat",
"title": "Fliplet.Chat",
- "description": "Build one-to-one, group, and public-channel chat features. Fliplet.Chat owns the conversations and messages data sources internally — supply only the contacts list (who can chat with whom).",
+ "description": "Chat API for V2 apps with a Chat component. Not available to V3 apps: build messaging on the app's own data sources.",
"docUrl": "https://developers.fliplet.com/API/fliplet-chat.md",
"preloaded": false,
"capabilities": [
@@ -103,7 +103,7 @@
"message history"
],
"category": "communications",
- "notes": "Owns the conversations and messages data sources internally — never propose `Conversations`, `Messages`, `Typing Status`, or similar data sources of your own. The only data source you design is the `contacts` list passed to `Fliplet.Chat.connect()` (who can chat with whom)."
+ "notes": "V3 apps can't use Fliplet.Chat: its server routes need a Chat component on a screen. Build messaging on the app's own data sources, with select rules that admit only the sender and the recipient."
},
{
"package": "fliplet-communicate",
@@ -133,7 +133,7 @@
"package": "fliplet-datasources",
"namespace": "Fliplet.DataSources",
"title": "Fliplet.DataSources",
- "description": "Connect to, query, insert, update, and delete records in Fliplet Data Sources from inside an app. All methods are promise-based.",
+ "description": "Connect to Fliplet data sources and find references for queries, writes, subscriptions, native storage and source management.",
"docUrl": "https://developers.fliplet.com/API/fliplet-datasources.md",
"preloaded": false,
"capabilities": [
diff --git a/docs/.well-known/llms.txt b/docs/.well-known/llms.txt
index 8a404ddc..dd22344c 100644
--- a/docs/.well-known/llms.txt
+++ b/docs/.well-known/llms.txt
@@ -84,9 +84,18 @@
## Data Sources
-- [Data Source joins](https://developers.fliplet.com/API/datasources/joins.md): Fetch related rows from multiple data sources in a single query using named joins, like SQL joins.
+- [Data Source file attachments](https://developers.fliplet.com/API/datasources/file-attachments.md): Attach files to Data Source entries with insert or update, handle upload constraints and distinguish file uploads from dataset imports.
+- [Data Source joins](https://developers.fliplet.com/API/datasources/joins.md): Fetch related rows with named join configuration and read them from each record’s joins property.
+- [Managing data sources](https://developers.fliplet.com/API/datasources/managing-data-sources.md): Source metadata, creation, configuration, deletion and Studio user roles, separate from app record access.
+- [Offline Data Source database](https://developers.fliplet.com/API/datasources/offline-database.md): Read and modify native bundled Data Sources with Database handles, local collection methods, views, indexes, and explicit persistence boundaries.
- [Data Sources query operators](https://developers.fliplet.com/API/datasources/query-operators.md): Filter Data Source queries with MongoDB/Sift operators ($eq, $gt, $in, $regex, $and, $or) inside connection.find() where clauses.
-- [Data Source views](https://developers.fliplet.com/API/datasources/views.md): Define named, session-aware filters on a data source so each user or group sees only the rows that apply to them.
+- [Reading Data Source records](https://developers.fliplet.com/API/datasources/reading-data.md): Read records, filter and sort queries, paginate with cursors, and retrieve distinct column values with Fliplet.DataSources.
+- [Data source security examples](https://developers.fliplet.com/API/datasources/security-examples.md): Pair public, shared, owner-filtered read, profile and role-based examples with data shapes and allowed and denied SDK operations.
+- [Data source security rule reference](https://developers.fliplet.com/API/datasources/security-rules.md): Access rule evaluation, trusted identity, requirement targets, column restrictions and custom script behavior.
+- [Data Source subscriptions](https://developers.fliplet.com/API/datasources/subscriptions.md): Subscribe to Data Source changes, handle entry payloads, and stop monitoring when a screen is hidden or removed.
+- [Testing data source security in preview](https://developers.fliplet.com/API/datasources/testing-security.md): Verify allowed and denied data source operations as authenticated app users with Studio preview security enforcement enabled.
+- [Data Source views](https://developers.fliplet.com/API/datasources/views.md): Request named filters that bind to the current login session. Views filter query results; access rules must enforce privacy independently.
+- [Writing Data Source records](https://developers.fliplet.com/API/datasources/writing-data.md): Insert, update, delete and commit Data Source records with the correct input shapes, options, write outcomes and security limits.
## Integrations
@@ -110,12 +119,12 @@
- [Fliplet.Media.Audio.Player](https://developers.fliplet.com/API/fliplet-audio-player.md): Embed an audio player UI in a screen by tagging HTML elements with audio URLs; the framework auto-initializes players on screen load.
- [Fliplet.Media.Audio](https://developers.fliplet.com/API/fliplet-audio.md): Play, pause, stop, and seek audio files on device or from a URL in Fliplet apps via the Audio namespace.
- [Fliplet.Barcode](https://developers.fliplet.com/API/fliplet-barcode.md): Scan QR codes and 1D/2D barcodes from the camera on web and native (attachScanner / scan) and generate barcode images, via the fliplet-barcode package.
-- [Fliplet.Chat](https://developers.fliplet.com/API/fliplet-chat.md): Build one-to-one, group, and public-channel chat features. Fliplet.Chat owns the conversations and messages data sources internally — supply only the contacts list (who can chat with whom).
+- [Fliplet.Chat](https://developers.fliplet.com/API/fliplet-chat.md): Chat API for V2 apps with a Chat component. Not available to V3 apps: build messaging on the app's own data sources.
- [Fliplet.Communicate](https://developers.fliplet.com/API/fliplet-communicate.md): Send email, SMS, push notifications, and share URLs from a Fliplet app using a single Communicate namespace.
- [Fliplet.Content()](https://developers.fliplet.com/API/fliplet-content.md): Create, query, update, and delete shared content records (bookmarks, likes, saved searches) backed by a data source via Fliplet.Content.
- [Fliplet.CSV](https://developers.fliplet.com/API/fliplet-csv.md): Encode and decode CSV in Fliplet apps via the fliplet-csv package.
- [Fliplet.Database](https://developers.fliplet.com/API/fliplet-database.md): Low-level helper for reading and querying JSON data from a local file as a database; used internally by Fliplet.DataSources.
-- [Fliplet.DataSources](https://developers.fliplet.com/API/fliplet-datasources.md): Connect to, query, insert, update, and delete records in Fliplet Data Sources from inside an app. All methods are promise-based.
+- [Fliplet.DataSources](https://developers.fliplet.com/API/fliplet-datasources.md): Connect to Fliplet data sources and find references for queries, writes, subscriptions, native storage and source management.
- [Fliplet.DataSources.Encryption](https://developers.fliplet.com/API/fliplet-encryption.md): Automatically encrypt and decrypt selected Data Source columns on-device by registering a private key and column list.
- [fliplet-error-tracking](https://developers.fliplet.com/API/fliplet-error-tracking.md): Auto-loading runtime that captures JavaScript exceptions and unhandled promise rejections and batches them to Fliplet.App.Logs.
- [Fliplet.Gamify](https://developers.fliplet.com/API/fliplet-gamify.md): Configure gamification with logs, variables, and achievements via Fliplet.Gamify; track points, milestones, and badges backed by a data source per user.
@@ -215,7 +224,7 @@
- [Fliplet infrastructure data flow](https://developers.fliplet.com/Data-flow.md): Architecture diagram showing how data flows between Fliplet Studio, Fliplet servers, app clients, and connected data sources.
- [Data integration service](https://developers.fliplet.com/Data-integration-service.md): Fliplet Agent (Data integration service) is a command line utility to synchronize data to and from Fliplet Servers.
- [Data Source Hooks](https://developers.fliplet.com/Data-Source-Hooks.md): Trigger emails, SMS, push notifications, web requests, or column operations on Data Source insert/update/beforeSave/beforeQuery using Sift.js match conditions.
-- [Securing Fliplet data sources](https://developers.fliplet.com/Data-source-security.md): Secure your Data Sources with access rules, data requirements, and custom JavaScript security rules.
+- [Securing Fliplet data sources](https://developers.fliplet.com/Data-source-security.md): Configure data source access rules and verify allowed and denied operations as authenticated app users.
- [Dependencies and assets](https://developers.fliplet.com/Dependencies-and-assets.md): Declare package dependencies and bundled JS/CSS/font assets for Fliplet components and themes, with separate interface and build entries plus appAssets.
- [Sending events between components](https://developers.fliplet.com/Event-emitter.md): Components interfaces can send events to Fliplet Studio using a event emitter bus provided with the `fliplet-core` dependency.
- [Fliplet app execution flow](https://developers.fliplet.com/Execution-flow.md): The rendering and hook lifecycle Fliplet apps go through before a screen is shown to the user, and where to safely run custom code within it.
diff --git a/docs/API/datasources/file-attachments.md b/docs/API/datasources/file-attachments.md
new file mode 100644
index 00000000..649d7081
--- /dev/null
+++ b/docs/API/datasources/file-attachments.md
@@ -0,0 +1,141 @@
+---
+title: Data Source file attachments
+description: "Attach files to Data Source entries with insert or update, handle upload constraints and distinguish file uploads from dataset imports."
+type: api-reference
+tags: [js-api, datasources, files]
+v3_relevant: true
+deprecated: false
+---
+# Data Source file attachments
+
+Single-record `insert` and `update` accept file attachments alongside column values. Uploaded fields become stored file URLs with associated media IDs; dataset import is a separate capability.
+
+## Before you start
+
+Load the [Data Sources library](../fliplet-datasources.md), configure app-user [write permissions](../../Data-source-security.md) and use a source with an attachment column. The examples use **Equipment requests** with `Item` and `Receipt` columns. The caller also needs the appropriate app/media context and available storage.
+
+The example requires a selected browser `File` from an ``. Omitted arguments default to empty objects.
+
+| `data` property | Type and behavior |
+|---|---|
+| `appId` | App identifier; the caller must have the necessary app access. |
+| `attributes` | Array of names or comma-separated string selecting source metadata fields. |
+| `include` | Comma-separated associated metadata names supported by the listing endpoint. |
+| `type` | Source type; an explicitly empty value requests ordinary sources. |
+| `excludeTypes` | Comma-separated source types to exclude; system-source exclusions also apply. |
+| `order`, `direction` | Metadata sort field and direction; default `name`, `ASC`. |
+| `roles` | Comma-separated Studio app role names, applied within the caller's listing context. These are not source `crudq` permission letters or app-login `Roles` fields. |
+| `definition` | Object filter on source definitions in the app context. The SDK serializes it and disables request caching. This is not an entry `where` query. |
+| `includeInUse` | String `'true'` requests in-use information in supported listing contexts. |
+| `includeCount` | Presence requests counts in supported listing contexts. |
+
+`options.cache` defaults to `true`; explicit `false` bypasses SDK request caching. Listing rejects on access, request or filter failures. Results are source objects such as `{ id, name, columns, definition }`, with fields determined by projection and permissions.
+
+```js
+async function listSourceNames() {
+ const sources = await Fliplet.DataSources.get({ attributes: ['id', 'name'] }, {
+ cache: false
+ });
+ return sources.map(source => ({ id: source.id, name: source.name }));
+}
+```
+
+## Read source metadata
+
+`Fliplet.DataSources.getById(id, options?)` returns `Promise` and rejects for missing/inaccessible sources or request failures.
+
+| Option | Type and behavior |
+|---|---|
+| `attributes` | Array or comma-separated string of metadata fields. |
+| `cache` | Request caching; default `true`. |
+| `params` | Query-parameter object. `includeEntriesCount` requests an entry count when present. `include: 'associatedPages'` requests page associations when an app context exists. |
+
+```js
+async function inspectSource(sourceId) {
+ return Fliplet.DataSources.getById(sourceId, {
+ attributes: ['id', 'name', 'columns'],
+ cache: false,
+ params: { includeEntriesCount: true }
+ });
+}
+```
+
+Reading source metadata does not prove permission to read every entry or bypass app rules.
+
+## Create a source
+
+`Fliplet.DataSources.create(options)` returns `Promise`. Use a nonempty `name` and supply the authorized app/organization scope. The SDK does not automatically add `appId` or `organizationId`; the request needs one of these scopes.
+
+| Option | Type and consequence |
+|---|---|
+| `name` | Required nonempty String. |
+| `appId` or `organizationId` | Required management scope identifier; supply the app or organization the caller can administer. |
+| `columns` | Array of column names. |
+| `entries` | Optional seed-entry array handled during source creation; returned value is the source, not a list of inserted rows. |
+| `accessRules` | Rule array. If omitted/falsy, the SDK supplies `[]`, denying app operations until an applicable grant is configured. |
+| `definition` | Source definition Object; login, view and index settings need their owning references. |
+| `hooks` | Server-hook configuration; see [Data Source Hooks](../../Data-Source-Hooks.md). |
+| `bundle` | Bundling setting for native apps. |
+| `encrypted`, `type`, `appCapabilities` | Source configuration fields for the relevant management workflow; do not infer app-user permissions from them. |
+
+This setup example creates an empty source with app access denied. Run it once in a disposable, authorized management context; repeated calls create additional sources rather than guaranteeing name-based reuse.
+
+```js
+async function createDisposableSource(appId) {
+ if (!appId) throw new Error('Supply the disposable test app ID.');
+ const source = await Fliplet.DataSources.create({
+ appId,
+ name: 'Workshop Test Records',
+ columns: ['Title', 'Status'],
+ accessRules: []
+ });
+ console.log('Created source', source.id);
+ return source;
+}
+```
+
+Invalid configuration, missing scope, feature limits or permission failures reject. Configure and verify [access rules](../../Data-source-security.md) before app users use the new source.
+
+## Update source settings
+
+`Fliplet.DataSources.update(id, data)` and `Fliplet.DataSources.update({ id, ...data })` return `Promise`. Missing ID throws synchronously in the online SDK path. The object overload removes `id` from the supplied object; pass a fresh object if you need to retain it.
+
+The source must allow the caller to update settings. Supported settings include `name`, `columns`, `definition`, `hooks`, `bundle`, `accessRules` and `appCapabilities`. Each field has its own setter/validation behavior; do not assume recursive merging or that this is a record update. A successful update clears SDK request caches. Invalid names/settings, denied permission or request failures reject.
+
+```js
+async function renameSource(sourceId) {
+ const source = await Fliplet.DataSources.update(sourceId, {
+ name: 'Workshop Archive'
+ });
+ return source.name;
+}
+```
+
+`connection.update(id, data, options?)` updates one entry instead. Its separate signature and outcomes are in [writing data](writing-data.md).
+
+## Delete a source
+
+`Fliplet.DataSources.delete(id)` returns a Promise resolving the deletion response. It deletes the source rather than an individual entry; do not expect an entry array. Use only a disposable source or a deliberate management action. App-token callers are rejected by the source-deletion endpoint. Permission and request failures reject.
+
+Delete one entry with `connection.removeById(id)` instead. Do not implement source deletion as app-user record cleanup.
+
+## Studio user roles
+
+These connection methods manage source permissions for Studio users. They do not create app login records, set a login record's `Roles` array or prove app-user authorization.
+
+| Method | Arguments and result |
+|---|---|
+| `connection.getUsers()` | Returns `Promise` of `{ fullName, firstName, lastName, email, dataSourceRole }`. Email is masked and `dataSourceRole.userId` is opaque, not a numeric account ID. |
+| `connection.addUserRole(data)` | `data.userId` is a numeric Studio account ID; `permissions` defaults to `'crudq'` and accepts only those permission letters; optional `definition` holds role configuration. Returns the created role object. |
+| `connection.removeUserRole(userId)` | Uses the source-role identifier expected by the management endpoint, which can be the opaque identifier from the listing. Returns the deletion response, not the removed user. |
+
+`dataSourceRole` includes `userId`, `permissions`, `definition`, `createdAt`, `updatedAt` and `dataSourceId`. Do not feed a masked email or opaque listing identifier into `addUserRole` as the numeric account ID. Invalid permission letters or a nonnumeric creation ID reject with a validation error; missing role/access and request failures reject.
+
+## Mapped-row helper
+
+`Fliplet.DataSources.fetchWithOptions(options)` is a convenience wrapper. Prefer `connection.find` when entry IDs/metadata or pagination envelopes are needed.
+
+| Option | Contract |
+|---|---|
+| `dataSourceId` | Required source identifier. Missing/falsy value rejects. |
+| `query` | Query object passed to `find`. Keep normal array results; pagination/object envelopes are not compatible with its row mapping. |
+| `columns` | Array of field names or single-key maps, e.g. `['Name', { label: 'Title' }]`. Omitted/empty selects row data directly. |
+| `compact` | Default `true`; excludes mapped null/undefined/blank strings. `false` retains these mapped values. |
+| `cache` | Converted to Boolean; default `false` for the entry request. |
+
+The helper connects online and normally resolves `{ dataSource, dataSourceEntries }`. `dataSourceEntries` contains flat column objects, not `{ id, data }` entries. `dataSource.columns` is recalculated from available mapped columns. Empty mapped rows are removed. A metadata failure is tolerated and can leave metadata incomplete; an entry query failure still rejects. When `query.aggregate` is supplied, it returns the raw aggregation result instead of this wrapper.
+
+## Client helpers and internal exports
+
+[The main reference](../fliplet-datasources.md#sdk-helper-boundaries) classifies client hooks, temporary IDs and connection lookup helpers. They do not enforce server authorization. Underscore helpers and core queue plumbing are not recommended app interfaces. Native `applyDiff` is described only as an update-pipeline compatibility surface in [offline storage](offline-database.md).
diff --git a/docs/API/datasources/offline-database.md b/docs/API/datasources/offline-database.md
new file mode 100644
index 00000000..98af5532
--- /dev/null
+++ b/docs/API/datasources/offline-database.md
@@ -0,0 +1,202 @@
+---
+title: Offline Data Source database
+description: "Read and modify native bundled Data Sources with Database handles, local collection methods, views, indexes, and explicit persistence boundaries."
+type: api-reference
+tags: [js-api, datasources, offline, native]
+v3_relevant: true
+deprecated: false
+---
+# Offline Data Source database
+
+`Fliplet.DataSources.Database` opens bundled Data Source data and returns local collection handles. Native collection writes change the local bundle; they do not make an online Data Source write or establish server authorization.
+
+## Choose the correct API
+
+Use [Data Source connections](../fliplet-datasources) for application reads and writes. Native connections integrate bundled reads and local acknowledgments with online operations. The lower-level `Database` API is for code that intentionally manages local bundle data.
+
+A native bundle and `Fliplet.Native.Maintenance` must be available. Opening a database on web fetches bundle data, but local write methods still call native file APIs. A successful web read does not establish native filesystem or synchronization behavior. Native applications must verify file persistence, incremental delivery and recovery on their supported devices.
+
+Offline data and cached-session views are not substitutes for [security-enabled online tests](testing-security).
+
+## Open a bundled Data Source
+
+`Fliplet.DataSources.Database(options)` returns `Promise`.
+
+| Option | Type | Required | Behavior |
+|---|---|---|---|
+| `id` | Number or String | Yes when not supplying `name` | Data Source ID; a string name requires `isName: true`. |
+| `isName` | Boolean | No | With a truthy value, resolves `name || id` from the app's bundled-source metadata. Use `true` for names. |
+| `name` | String | No | Alternative name when `isName: true`; it takes precedence over `id`. |
+| `appId` | Number | No | Defaults to the current app ID. Selects the native app data directory. |
+
+The following native example assumes `Products` is bundled in the app. It reads without changing the bundle.
+
+```js
+const database = await Fliplet.DataSources.Database({
+ id: 'Products',
+ isName: true
+});
+const products = database.dataSource(null, { name: 'Products' });
+const entries = await products.find({}, {});
+for (const entry of entries) {
+ console.log(entry.id, entry.data.Name);
+}
+```
+
+An ID-based connection uses matching IDs at both stages:
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+const entry = await products.findById(101);
+if (entry) console.log(entry.data.Name);
+```
+
+The filename-string overload, such as `Database('dataSources.db')`, supports older multi-source bundles. It is a compatibility form used by the connection API. Prefer the ID/name options for dedicated bundles; do not assume a dedicated bundle includes other Data Sources.
+
+There is no public close/dispose method on a database handle. Do not depend on repeated native opens avoiding file reads; use the handle you already opened for related operations.
+
+## Select a collection
+
+`database.dataSource(id, options?)` returns a collection handle synchronously.
+
+| Argument | Type | Behavior |
+|---|---|---|
+| `id` | Number or numeric String | Selects a Data Source present in the loaded bundle. |
+| `options.name` | String | Resolves the name within the loaded bundle's source descriptors instead of using the supplied ID. |
+
+Selecting an unavailable source can throw synchronously. Choose a source actually present in the opened bundle and wrap selection as well as awaited operations in error handling.
+
+Collection entries have metadata and a nested data object:
+
+```js
+const exampleEntry = { id: 101, data: { Name: 'USB-C hub', Category: 'Accessories', Price: 49.99 } };
+```
+
+## Read local entries
+
+### `find(where, options)`
+
+Returns `Promise>`. Both arguments should be supplied: use `{}` for an empty filter and `{}` for no options. Omitting `options` can throw synchronously.
+
+`where` filters the entry's `data` fields using the local Sift matcher. This API does not inherit every online query option. It does not provide pagination counts, joins, online authorization or server-side sorting.
+
+| Option | Type | Default | Behavior |
+|---|---|---|---|
+| `limit` | Number | No limit | A truthy value limits the matched entries. `0` does not limit results. |
+| `views` | Array of strings | None | Applies matching named views before the filter. Multiple resolved view filters are combined with OR. |
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+const affordableAccessories = await products.find({
+ Category: 'Accessories',
+ Price: { $lte: 50 }
+}, { limit: 20 });
+console.log(affordableAccessories.map(entry => entry.data.Name));
+```
+
+Views resolve values from cached Data Source session data and, when available, the device UUID. Unknown view names or missing view definitions can leave results unfiltered. Treat views as local display filtering, not as an access-control boundary. See [Data Source views](views).
+
+### `findById(id)`
+
+Returns `Promise`. The filter addresses the entry's metadata `id`, not a column named `ID`.
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+const product = await products.findById(101);
+if (product) console.log(product.data.Name);
+else console.log('Product is not in the local bundle.');
+```
+
+## Write the local bundle
+
+Each method below changes in-memory data before writing the full JSON bundle to native storage. A file-write failure rejects the promise; it does not undo the earlier memory change. Local writes do not send server updates, validate server rules, or generate server IDs. Use [online writes](writing-data) when the server must persist the change.
+
+| Method | Arguments | Resolved value |
+|---|---|---|
+| `insert(entry)` | Complete entry object, including its ID and `data` | The supplied entry object. |
+| `update(existingEntry, patch)` | A fetched collection entry and a partial entry object | The supplied patch, not the complete merged entry. |
+| `remove(entryOrPredicate)` | Entry object matched by `id`, or a function returning true for entries to remove | The supplied object or predicate function, not a list of removed entries. |
+
+`update` deep-merges the patch into the existing entry in place. These operations are local bookkeeping; an ID supplied to `insert` does not reserve that ID on the server.
+
+This example deliberately changes a local copy of an existing product. Verify device persistence before using this behavior in an application workflow.
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+const product = await products.findById(101);
+if (!product) throw new Error('Product 101 is missing from the local bundle.');
+
+const patch = { data: { Price: 44.99 } };
+const result = await products.update(product, patch);
+console.log(result.data.Price); // 44.99: result is the patch
+console.log(product.data.Name); // Existing fields remain on the merged entry
+```
+
+For predicate removal, inspect the affected rows first:
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+const discontinued = await products.find({ Status: 'Discontinued' }, {});
+console.log('Local entries selected for removal:', discontinued.map(entry => entry.id));
+// Run only when the application intends to remove these local entries:
+// await products.remove(entry => entry.data.Status === 'Discontinued');
+```
+
+## Apply a server diff
+
+`collection.applyDiff(diff)` returns `Promise`. This method supports the native update pipeline; it is not an online write or a custom synchronization service.
+
+| Field | Type | Behavior |
+|---|---|---|
+| `updated` | Array of entries | Required in incremental mode. Removes existing entries with the same IDs, then appends the supplied replacements. |
+| `deleted` | Array of objects with `id` | Required in incremental mode. Removes matching IDs. |
+| `fullRefresh` | Boolean | When truthy, replaces the collection with `updated` or an empty array if `updated` is absent. |
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+// Example diff for a disposable native fixture, not an online server write.
+await products.applyDiff({
+ updated: [{ id: 101, data: { Name: 'USB-C hub', Price: 49.99 } }],
+ deleted: [{ id: 103 }]
+});
+```
+
+A full refresh is destructive to local entries absent from the supplied array. Incremental replacements do not deep-merge old entry data.
+
+## Read local indexes
+
+`indexes(columns)` returns `Promise` mapping each requested column to unique values. `index(column)` returns `Promise` for one column.
+
+Precomputed bundle indexes are used when present. Otherwise values are computed from the collection: array values are flattened, and falsy values such as `false`, `0`, `''`, `null` and `undefined` are omitted. A computed index is not a complete enumeration of all boolean or numeric values.
+
+```js
+const database = await Fliplet.DataSources.Database({ id: 42 });
+const products = database.dataSource(42);
+const categories = await products.index('Category');
+console.log(categories); // Example: ['Accessories', 'Lighting']
+const values = await products.indexes(['Category', 'Status']);
+console.log(values.Category);
+```
+
+Empty collections, missing indexes and unavailable collection data can reject. A missing single-column index is not guaranteed to resolve to `[]`. Precomputed indexes are not recomputed by the mutation methods described above; do not assume they reflect a subsequent local edit.
+
+## Errors and recovery
+
+| Operation | Failure behavior |
+|---|---|
+| Open by name | Rejects when the name cannot be resolved from the app's bundled-source metadata. |
+| Open native file | Rejects for missing data or invalid JSON, and propagates native read failures. |
+| Select a collection | Can throw immediately if the source is absent from the loaded bundle. |
+| Read/write invalid collection | Rejects when collection entries are unavailable. `find` can also throw when options are omitted. |
+| Persist a local mutation | Rejects native write failures after the memory change. |
+| Read an index | Rejects unavailable or empty index results as described above. |
+
+Error messages are localized; do not match their English text for application control flow. When a local write fails, report the failure and reload authoritative data through the application's established update flow rather than treating the in-memory change as durable.
+
+[Data Sources API](../fliplet-datasources) · [Subscriptions](subscriptions) · [Testing security](testing-security)
diff --git a/docs/API/datasources/query-operators.md b/docs/API/datasources/query-operators.md
index 4c888602..102ed006 100644
--- a/docs/API/datasources/query-operators.md
+++ b/docs/API/datasources/query-operators.md
@@ -20,10 +20,10 @@ Fliplet Data Sources uses [Sift.js](https://github.com/Fliplet/sift.js) which pr
|----------|-------------|---------|-------|
| `$eq` | Equal to | `{ age: { $eq: 25 } }` | Can be simplified to `{ age: 25 }` |
| `$ne` | Not equal to | `{ status: { $ne: 'inactive' } }` | |
-| `$gt` | Greater than | `{ score: { $gt: 80 } }` | Optimized for performance |
-| `$gte` | Greater than or equal | `{ score: { $gte: 80 } }` | Optimized for performance |
-| `$lt` | Less than | `{ age: { $lt: 30 } }` | Optimized for performance |
-| `$lte` | Less than or equal | `{ age: { $lte: 30 } }` | Optimized for performance |
+| `$gt` | Greater than | `{ score: { $gt: 80 } }` | |
+| `$gte` | Greater than or equal | `{ score: { $gte: 80 } }` | |
+| `$lt` | Less than | `{ age: { $lt: 30 } }` | |
+| `$lte` | Less than or equal | `{ age: { $lte: 30 } }` | |
```js
// Examples
@@ -43,7 +43,7 @@ const highScores = await connection.find({
| Operator | Description | Example | Notes |
|----------|-------------|---------|-------|
-| `$in` | Value in array | `{ category: { $in: ['tech', 'science'] } }` | Optimized for performance |
+| `$in` | Value in array | `{ category: { $in: ['tech', 'science'] } }` | |
| `$nin` | Value not in array | `{ status: { $nin: ['banned', 'suspended'] } }` | |
| `$all` | Array contains all values | `{ tags: { $all: ['urgent', 'review'] } }` | For array fields |
| `$size` | Array size equals | `{ items: { $size: 3 } }` | For array fields |
@@ -66,8 +66,8 @@ const completeTasks = await connection.find({
| Operator | Description | Example | Notes |
|----------|-------------|---------|-------|
-| `$and` | Logical AND | `{ $and: [{ age: { $gte: 18 } }, { status: 'active' }] }` | Optimized for performance |
-| `$or` | Logical OR | `{ $or: [{ role: 'admin' }, { role: 'moderator' }] }` | Optimized for performance |
+| `$and` | Logical AND | `{ $and: [{ age: { $gte: 18 } }, { status: 'active' }] }` | |
+| `$or` | Logical OR | `{ $or: [{ role: 'admin' }, { role: 'moderator' }] }` | |
| `$nor` | Logical NOR | `{ $nor: [{ status: 'banned' }, { status: 'suspended' }] }` | None of the conditions |
| `$not` | Logical NOT | `{ age: { $not: { $lt: 18 } } }` | Negates the condition |
@@ -98,8 +98,8 @@ const activeUsers = await connection.find({
| Operator | Description | Example | Notes |
|----------|-------------|---------|-------|
-| `$regex` | Regular expression | `{ email: { $regex: /@company\.com$/i } }` | Case-insensitive with `i` flag |
-| `$iLike` | Case-insensitive partial match | `{ name: { $iLike: 'john' } }` | Fliplet-specific, optimized |
+| `$regex` | Regular expression | `{ email: { $regex: '@company\\.com$', $options: 'i' } }` | Case-insensitive with `i` flag |
+| `$iLike` | Case-insensitive partial match | `{ name: { $iLike: 'john' } }` | Fliplet-specific |
```js
// Example
@@ -111,10 +111,12 @@ const johnUsers = await connection.find({
### Existence and Type Operators
+Online queries use JSON. A JavaScript `RegExp` object becomes `{}` during serialization; use a string `$regex` and `$options` instead. The bundled `$type` implementation expects constructors, which are not JSON-safe, rather than strings such as `number`.
+
| Operator | Description | Example | Notes |
|----------|-------------|---------|-------|
| `$exists` | Field exists | `{ phone: { $exists: true } }` | Checks if field is present |
-| `$type` | Field type check | `{ score: { $type: 'number' } }` | Types: string, number, boolean, array, object |
+| `$type` | Not supported as an online JSON type-name filter | — | Do not pass string type names or constructors in online queries. |
```js
// Examples
@@ -122,9 +124,7 @@ const usersWithPhone = await connection.find({
where: { phone: { $exists: true } }
});
-const numericScores = await connection.find({
- where: { score: { $type: 'number' } }
-});
+// Validate field types in application code; $type is not a JSON type-name filter.
```
### Mathematical Operators
@@ -134,7 +134,7 @@ const numericScores = await connection.find({
| `$mod` | Modulo operation | `{ id: { $mod: [2, 0] } }` | `[divisor, remainder]` |
```js
-// Examples - find even IDs
+// Find even values in the data column named id (not record metadata IDs)
const evenIds = await connection.find({
where: { id: { $mod: [2, 0] } }
});
@@ -164,7 +164,7 @@ const studentsWithGoodGrades = await connection.find({
## Fliplet Custom $filters Operator
-Fliplet provides a custom `$filters` operator that offers optimized performance and additional conditions not available in standard MongoDB operators.
+Fliplet provides a custom `$filters` operator for named filter conditions. Filters are combined with AND. Online database optimization depends on the expression and input values; nested filters may be evaluated in memory.
### Syntax
@@ -194,6 +194,10 @@ Fliplet provides a custom `$filters` operator that offers optimized performance
| `<` | Less than | Number | `{ column: 'Price', condition: '<', value: 100 }` |
| `<=` | Less than or equal | Number | `{ column: 'Quantity', condition: '<=', value: 50 }` |
| `contains` | Case-insensitive partial match | String | `{ column: 'Email', condition: 'contains', value: '@company.com' }` |
+| `notcontain`, `notcontains` | Excludes matching text; missing values do not count as a negative match | String | `{ column: 'Email', condition: 'notcontains', value: '@example.com' }` |
+| `regex` | Regular-expression string | String | `{ column: 'Email', condition: 'regex', value: '@example[.]com$' }` |
+| `notoneof`, `notin` | Excludes values from a list | Array/String | `{ column: 'Status', condition: 'notin', value: ['Archived'] }` |
+| `none` | Inactive filter; matches all rows | None | `{ column: 'Status', condition: 'none' }` |
| `empty` | Field is empty | None | `{ column: 'Notes', condition: 'empty' }` |
| `notempty` | Field is not empty | None | `{ column: 'Description', condition: 'notempty' }` |
| `between` | Numeric range (inclusive) | Object | `{ column: 'Age', condition: 'between', value: { from: 18, to: 65 } }` |
@@ -213,7 +217,7 @@ Fliplet provides a custom `$filters` operator that offers optimized performance
For date conditions, you can optionally specify a unit of comparison:
```js
-{
+const birthdayFilter = {
column: 'Birthday',
condition: 'dateis',
value: '1990-01-01',
@@ -287,12 +291,12 @@ const getFilteredUsers = async (filters = {}) => {
};
// Usage with destructuring
-const { length: userCount, ...users } = await getFilteredUsers({
+const users = await getFilteredUsers({
minAge: 25,
departments: ['Engineering', 'Design']
});
-console.log(`Found ${userCount} users matching criteria`);
+console.log(`Found ${users.length} users matching criteria`);
```
---
@@ -301,43 +305,11 @@ console.log(`Found ${userCount} users matching criteria`);
### Optimized Operators
-The following operators are optimized for better performance with Fliplet's database:
-
-**MongoDB-style (optimized):**
-- `$or`, `$and`, `$gte`, `$lte`, `$gt`, `$lt`, `$eq`, `$in`
-
-**Fliplet $filters (optimized):**
-- `==`, `contains` (especially optimized)
-
-**Optimized Value Types:**
-- Strings and numbers perform better than complex objects
+Some simple `$or`, `$and`, comparison and `$in` expressions can be evaluated by the online database. Supported `$filters` conditions can also be promoted when their structure and values permit. Other expressions use in-memory filtering. Operator choice alone does not guarantee a particular execution plan or faster query.
### Best Practices
-1. **Use $filters for complex conditions** - Better performance than equivalent MongoDB operators
-2. **Prefer optimized operators** - Use `$gte` instead of `$not: { $lt: value }`
-3. **Index-friendly queries** - Simple equality and range queries perform best
-4. **Combine efficiently** - Use `$and` for multiple conditions on different fields
-
-```js
-// Good - optimized query
-const optimized = await connection.find({
- where: {
- $filters: [
- { column: 'Status', condition: '==', value: 'Active' },
- { column: 'Score', condition: '>=', value: 80 }
- ]
- }
-});
-
-// Also good - using optimized MongoDB operators
-const mongoOptimized = await connection.find({
- where: {
- Status: 'Active',
- Score: { $gte: 80 }
- }
-});
-```
+Use the operator that expresses the required result. Add a positive `limit` for bounded reads and verify representative data and result counts. Combining filters does not turn a caller-controlled query into an access rule.
---
@@ -387,34 +359,26 @@ const userData = await getComplexUserData({
## Using operators in custom security rules
-When writing [custom security rules](/Data-source-security#custom-security-rules), you can query other Data Sources using the `DataSources` server-side library. The `find` and `findOne` methods support the same query operators listed above.
+The `DataSources(idOrName)` library inside [custom security scripts](security-rules#custom-scripts) has a separate lookup contract. Its `find(options?)` returns flat data objects, not SDK `{ id, data }` records; `findOne(options?)` returns one flat object or `undefined`.
+
+| Property | Type / default | Behavior |
+|---|---|---|
+| `where` | Object; omitted | Filters data fields. Scalar operator operands are restricted to `$or`, `$and`, `$gt`, `$gte`, `$lt`, `$lte`, `$ne`, `$not`, `$like`, `$iLike`, `$notLike`, `$notILike`, `$eq`, `$contains`. Do not assume the app Sift operators or `$filters` work here. |
+| `limit` | Number; at most `100` | Maximum lookup records; `findOne` sets it to `1`. |
+| `offset` | Number; omitted | Skip records. |
```js
-// Custom security rule: grant access if user is a manager in the same office
-if (type === 'select') {
- var entry = await DataSources(123).findOne({
- where: {
- Office: user.Office,
- Managers: { $in: [user.Email] },
- Status: { $ne: 'Inactive' }
- }
+// Security script: user is a flat login record and may be absent.
+if (user && user.Office && user.Email) {
+ const manager = await DataSources(123).findOne({
+ where: { Office: user.Office, Email: user.Email, Status: 'Active' }
});
-
- if (entry) {
- return { granted: true };
- }
+ if (manager) { return { granted: true }; }
}
+return { granted: false };
```
-Both `find` and `findOne` accept:
-
-| Property | Type | Default | Description |
-|---|---|---|---|
-| `where` | Object | — | Query filter using any operators from this page |
-| `limit` | Number | `100` | Maximum number of records to return |
-| `offset` | Number | `0` | Number of records to skip |
-
-The operators on this page are for querying data . The require property in security rules uses a different set of requirement types (equals, notequals, contains) to validate incoming queries — see data requirements and query validation .
+Array operands and complex nested expressions must be checked against this lookup contract independently; the whitelist is not a promise of full MongoDB compatibility. A missing or inaccessible source can throw. The rule `require` property uses requirement types such as `equals`, `notequals` and `contains`, described in [security rules](security-rules); it does not use this query operator table.
[Back to Data Sources Documentation](../fliplet-datasources)
-{: .buttons}
\ No newline at end of file
+{: .buttons}
diff --git a/docs/API/datasources/reading-data.md b/docs/API/datasources/reading-data.md
new file mode 100644
index 00000000..da94f11a
--- /dev/null
+++ b/docs/API/datasources/reading-data.md
@@ -0,0 +1,163 @@
+---
+title: Reading Data Source records
+description: "Read records, filter and sort queries, paginate with cursors, and retrieve distinct column values with Fliplet.DataSources."
+type: api-reference
+tags: [js-api, datasources, query]
+v3_relevant: true
+deprecated: false
+---
+# Reading Data Source records
+
+A Data Source connection reads records with `find`, `query`, `findOne` and `findById`. Query options control filtering and result shape; source permissions and [security rules](security-rules) control access.
+
+The examples use an existing source with numeric ID `123`. Replace this ID with the source attached to the app. Online calls can reject for inaccessible sources, invalid input, access rules or transport failures. Studio sample data is not evidence that app-user access works; verify in [preview with security enabled](testing-security).
+
+## Record shape
+
+Normal query results are arrays of records with `id` and `data`. Metadata such as `order`, `createdAt` and `updatedAt` may also be present. Aggregation can change this shape.
+
+```js
+// Example record, not an insert payload
+const record = { id: 41, data: { Name: 'Mina', Department: 'Design', Score: 86 } };
+console.log(record.id, record.data.Name);
+```
+
+## find and query
+
+`connection.find(query?, options?)` returns a Promise. `connection.query(query?, options?)` is an alias with the same arguments. Both default their arguments to `{}`. For online query updates and deletion, see [writing records](writing-data).
+
+| Query property | Type / default | Behavior |
+|---|---|---|
+| `where` | Object; omitted | Filters fields within `record.data`; see [query operators](query-operators). An omitted filter selects all accessible records. |
+| `id`, `createdAt`, `updatedAt` | Number/date or query object; omitted | Online filters on record metadata, outside `where`. For incremental reads, `updatedAt` can also return deleted-entry metadata in an object response. Not applied by native local reads. |
+| `dataSourceEntryId` | Number; omitted | Online filter on the record's metadata ID. `where: { id: ... }` instead filters a data column named `id`. |
+| `type` | String; omitted | Omitted or `select` reads records. Online `update` and `delete` are writes; see [writing records](writing-data). Native local reads do not implement the online query-update path. |
+| `data` | Object; omitted | Patch for online `type: 'update'`. Do not supply this for ordinary reads. |
+| `attributes` | String[]; omitted | Returns the listed data fields, subject to security field restrictions. Does not change metadata ID. |
+| `limit` | Number; omitted | Maximum records. Supply a positive integer; `0` is treated as omitted rather than an empty page. Online negative or nonnumeric values reject; fractions are rounded down. |
+| `offset` | Number; omitted | Skip records. Use a nonnegative integer; online negative or nonnumeric values reject. |
+| `order` | Array of pairs; online default `[['order', 'ASC'], ['id', 'DESC']]` | See [sorting](#sorting-and-ordering). Native local queries do not apply the same online ordering pipeline. |
+| `includePagination` | Boolean; omitted | Online literal `true` returns the root response with `entries` and `pagination`. Native local reads still return an array. |
+| `distinct` | String; omitted | Online retains the first row for each value of the named data column, after pagination. Other fields come from that row. Not applied by native local reads. |
+| `views` | String[]; omitted | Requests named [views](views); use this plural array form. Views do not replace security rules. |
+| `join` | Object; omitted | Named [joins](joins). Online and native implementations differ; verify the target platform. |
+| `aggregate` | Array; omitted | Mingo aggregation pipeline applied to records. Stage paths refer to record properties such as `data.Score`. Native execution requires Mingo to be loaded. |
+
+| Request option | Type / default | Behavior |
+|---|---|---|
+| `cache` | API cache option; omitted | Forwarded to the online API request; use `false` when fresh verification is needed. Native reads use the loaded local database. |
+| `authenticate` | Boolean; true when `Fliplet.Page` exists | Applies SDK entry authentication in supported paths. This is not a switch for server security enforcement. Delete queries turn it off. |
+| `returnAsObject` | Boolean; omitted | Online returns `{ entries, deletedEntries }`, taking precedence over `includePagination`. Native reads still return an array. |
+| `ack` | Boolean; omitted | Relevant to query deletion and native reconciliation; see [writing records](writing-data). |
+
+```js
+const connection = await Fliplet.DataSources.connect(123);
+const records = await connection.find({
+ where: { Department: 'Design', Score: { $gte: 80 } },
+ attributes: ['Name', 'Score'],
+ limit: 20,
+ order: [['data.Score', 'DESC']]
+}, { cache: false });
+records.forEach(function(record) { console.log(record.id, record.data.Name); });
+```
+
+### Pagination results
+
+Online `includePagination: true` returns `{ entries, dataSourceId, count, source, pagination, indexes, deletedEntries }`. Some optional properties may be absent. `count` is the returned array length. `pagination` contains `total`, `limit` and `offset`; omitted limits/offsets may be absent. `total` is calculated before distinct and required-join filtering, so it can exceed the final returned record count.
+
+```js
+const page = await connection.find({ limit: 20, offset: 0, includePagination: true });
+console.log(page.entries.length, page.pagination.total);
+```
+
+Do not combine this envelope with `findOne` or `findWithCursor`, which expect array-producing queries. `returnAsObject` produces a different envelope without pagination metadata.
+
+## findOne
+
+`connection.findOne(query?, options?)` sets `query.limit` to `1` and resolves the first record or `undefined`. It modifies the supplied query object. With online `returnAsObject: true`, it resolves `{ entries: recordOrUndefined, deletedEntries }`; `entries` is a single record here, not an array. Avoid combining `returnAsObject` with native local reads.
+
+```js
+const record = await connection.findOne({ where: { Email: 'mina@example.com' } });
+if (record) { console.log(record.id, record.data.Email); }
+```
+
+## findById
+
+`connection.findById(id)` resolves a single record. Online missing records reject with a 404 response and invalid IDs reject with a 400 response. Native local lookup can resolve `undefined`. The authenticated `session` identifier is reserved for session lookup; use numeric record IDs for ordinary reads. This method has no second request-options argument.
+
+```js
+try {
+ const record = await connection.findById(41);
+ if (record) { console.log(record.data.Name); }
+} catch (error) {
+ // Handle missing, denied or failed online requests in the app UI.
+ console.error(error);
+}
+```
+
+## Sorting and ordering
+
+Top-level `order` is an array of `[column, direction]` pairs. Metadata columns are `id`, `order`, `createdAt`, `deletedAt` and `updatedAt`. Data columns use a `data.` prefix and must exist on the source. Use `ASC` or `DESC`; online data-column directions are normalized to uppercase and reject other values. Data-column numeric-looking values are sorted numerically, while other values use text ordering. A join's `order` instead takes one flat pair.
+
+```js
+const records = await connection.find({
+ order: [['data.Department', 'ASC'], ['data.Score', 'DESC']]
+});
+```
+
+## findWithCursor
+
+`connection.findWithCursor(query?, options?)` resolves an array with cursor properties and methods after fetching its first page. Supply a positive integer `limit` and an array-producing query. Cursor movement changes the query synchronously; `update()` performs the next read.
+
+| Cursor member | Behavior |
+|---|---|
+| `query` | Mutable query; pagination is `cursor.query.limit` / `cursor.query.offset`, not `cursor.limit` / `cursor.offset`. |
+| `currentPage` | Zero-based page index, initially `0`. |
+| `isFirstPage`, `isLastPage` | Flags updated during movement/fetch. Last-page detection fetches one extra record. |
+| `firstRecordInNextPage` | Extra fetched record when another page exists; otherwise undefined. |
+| `idSet`, `containsId(id)` | Set of fetched IDs and membership check. |
+| `next()`, `prev()`, `setPage(page)` | Mutate page/offset and return the cursor. They do not fetch. Negative page/offset is clamped to zero. |
+| `update({ keepExisting? })` | Promise resolving the cursor. Default replaces its rows and ID set. `keepExisting: true` appends rows without deduplicating overlapping pages. |
+
+```js
+const cursor = await connection.findWithCursor({
+ limit: 20, order: [['id', 'ASC']]
+});
+console.log(cursor.length, cursor.query.offset);
+if (!cursor.isLastPage) {
+ await cursor.next().update();
+ console.log(cursor.currentPage, cursor.map(function(record) { return record.id; }));
+}
+```
+
+## Column indexes
+
+`connection.getIndex(column)` resolves an array of distinct values. `connection.getIndexes(columns)` resolves an object mapping each requested column to its values. These source-wide lookups do not accept the `where`, views, pagination or request options of `find`; for values that depend on the signed-in user, use `find` with `attributes`.
+
+| Argument | Type | Required |
+|---|---|---|
+| `column` | String | Yes for `getIndex`. |
+| `columns` | String[] | Yes for `getIndexes`; use a nonempty array. |
+
+Online values are extracted as text, with missing/null values represented as `null`; for example numeric `12` can return `'12'`. When the source declares columns, requesting an undeclared column rejects with a 400 response. Native indexes may use precomputed values or scan local rows; computed native indexes skip falsy values, including `false`, `0` and `''`. See [offline database](offline-database).
+
+```js
+const departments = await connection.getIndex('Department');
+const indexes = await connection.getIndexes(['Department', 'Status']);
+console.log(departments, indexes.Department, indexes.Status);
+```
+
+## Aggregation
+
+Aggregation operates on records, so field paths include `data.`. The result follows the supplied pipeline rather than guaranteeing `{ id, data }` records. Invalid pipelines can reject the query.
+
+```js
+const summary = await connection.find({
+ aggregate: [
+ { $group: { _id: '$data.Department', count: { $sum: 1 } } }
+ ]
+});
+console.log(summary); // [{ _id: 'Design', count: ... }, ...]
+```
+
+[Data Sources API](../fliplet-datasources)
diff --git a/docs/API/datasources/security-examples.md b/docs/API/datasources/security-examples.md
new file mode 100644
index 00000000..13b729d1
--- /dev/null
+++ b/docs/API/datasources/security-examples.md
@@ -0,0 +1,302 @@
+---
+title: Data source security examples
+description: "Pair public, shared, owner-filtered read, profile and role-based examples with data shapes and allowed and denied SDK operations."
+type: guide
+tags: [js-api, datasources, security]
+v3_relevant: true
+deprecated: false
+---
+
+# Data source security examples
+
+Use these policies with their stated data shapes and operation limits. Each example pairs the complete rule array with app SDK calls and outcomes to verify with security enforcement enabled.
+
+## Before you start
+
+Configure each data source's columns and sample rows in Studio before running the app code. For authenticated examples, configure a **Members** login source using the [authentication guide](../v3/auth.md). Replace illustrative ID `731` in every bound rule with its actual ID. Seed two test identities whose flat authenticated data includes:
+
+```json
+[
+ { "Email": "ada@example.org", "Enabled": true, "Roles": ["Author", "Approver"] },
+ { "Email": "lin@example.org", "Enabled": true, "Roles": ["Author"] }
+]
+```
+
+`Enabled` is a native boolean, and `Roles` is an array of strings. Protect those identity fields from self-service writes. Columns are names, not enforced types: verify actual stored values. Use disposable test sources and record allowed and denied results using the [preview procedure](testing-security.md). Outcomes below describe the policy contract, not a substitute for authenticated runtime tests.
+
+SDK query results have `{ id, data }` envelopes. Obtain current fixture IDs from Studio rather than guessing IDs. These plain JavaScript examples use `await` and run inside an app with the Data Sources library available. Authentication happens through the app's configured login flow before these calls.
+
+## Public lookup
+
+**Goal:** Anonymous readers can browse a public instrument catalog. App users cannot change it.
+
+Create **Instrument Catalog** with columns `Instrument` and `Family`, seeded with:
+
+```json
+[
+ { "Instrument": "Oboe", "Family": "Woodwind" },
+ { "Instrument": "Cello", "Family": "Strings" }
+]
+```
+
+Save this entire rule array:
+
+```json
+[
+ { "type": ["select"], "allow": "all" }
+]
+```
+
+Allowed as an anonymous app user:
+
+```js
+const catalog = await Fliplet.DataSources.connectByName('Instrument Catalog', { offline: false });
+const instruments = await catalog.find();
+console.log(instruments.map(entry => entry.data.Instrument));
+// Expected: ['Oboe', 'Cello'] (order is illustrative).
+```
+
+Denied as either an anonymous or signed-in app user:
+
+```js
+const catalog = await Fliplet.DataSources.connectByName('Instrument Catalog', { offline: false });
+let denied = false;
+try {
+ await catalog.insert({ Instrument: 'Flute', Family: 'Woodwind' });
+} catch (error) {
+ denied = true;
+ console.log(error); // Inspect the denial; then confirm no Flute row persisted.
+}
+if (!denied) throw new Error('Unexpected grant: verify the saved policy and preview mode.');
+```
+
+Update and delete are also denied because no rule grants them. Studio administrators may still seed the catalog; that success is separate from app-user evidence.
+
+## Shared signed-in notices
+
+**Goal:** Members from the intended login source can read and add shared notices. They cannot update or delete existing notices.
+
+Create **Rehearsal Notices** with columns `Message` and `Venue`, seeded with `{ "Message": "Bring a music stand", "Venue": "Hall B" }`. Save:
+
+```json
+[
+ {
+ "type": ["select"],
+ "allow": { "loggedIn": true, "dataSourceId": 731 }
+ },
+ {
+ "type": ["insert"],
+ "allow": { "loggedIn": true, "dataSourceId": 731 },
+ "require": ["Message", "Venue"]
+ }
+]
+```
+
+Allowed after Ada signs in through Members:
+
+```js
+const notices = await Fliplet.DataSources.connectByName('Rehearsal Notices', { offline: false });
+await notices.insert({ Message: 'Doors open at six', Venue: 'Hall A' });
+const rows = await notices.find({ where: { Venue: 'Hall A' } });
+console.log(rows.map(entry => entry.data));
+```
+
+Denied tests: sign out and call `find()` or `insert()`; sign in through another login source and repeat. As Ada, submit `insert({ Message: 'Missing venue' })`, or update/delete a seeded notice. Each must reject and leave no forbidden persisted effect. The string requirements check presence only: `Venue: null` is not a nonempty validation failure under this policy.
+
+`
-**When to use:** Preferred method - data source names remain consistent when copying apps between environments
-
-```js
-const connection = await Fliplet.DataSources.connectByName("Users");
-```
-
-**Why use connectByName:** Data source names typically remain the same when copying apps between environments, while IDs change. This makes your code more portable and maintainable.
-
-### Connect to a Data Source by ID
-
-**Purpose:** Establish a connection to work with a specific data source by ID
-**Syntax:** `Fliplet.DataSources.connect(dataSourceId, options?)`
-**Returns:** `Promise`
-**When to use:** Only when you specifically need to target a data source by ID (less portable)
-
-```js
-// Basic connection by ID (not recommended for portability)
-const connection = await Fliplet.DataSources.connect(123);
-
-// Advanced connection with options
-const connection = await Fliplet.DataSources.connect(123, {
- offline: false // Force online-only queries (mobile apps default to offline)
-});
-```
-
-**Note:** Fliplet apps on mobile devices attempt to connect to the **offline bundled data sources by default**. You can optionally prevent a data source from being bundled by editing its settings in Fliplet Studio, or by using the `offline: false` parameter to connect only to the live online data source.
-
----
-
-## Essential Connection Methods
-
-### 1. Find Records (Query Data)
-
-**Purpose:** Retrieve records from the data source
-**Syntax:** `connection.find(options?)`
-**Returns:** `Promise`
-**When to use:** Reading/querying data (most common operation)
-
-#### Basic Usage
-```js
-// Complete example: Get all user records
-const connection = await Fliplet.DataSources.connectByName("Users");
-const allUsers = await connection.find();
-console.log(`Found ${allUsers.length} users:`, allUsers);
-
-// Complete example: Get users from London office
-const connection = await Fliplet.DataSources.connectByName("Users");
-const londonUsers = await connection.find({
- where: { Office: 'London' }
-});
-console.log('London users:', londonUsers);
-
-// Complete example: Get specific user data columns only
-const connection = await Fliplet.DataSources.connectByName("Users");
-const userNames = await connection.find({
- attributes: ['Name', 'Email', 'Department']
-});
-console.log('User names and emails:', userNames);
-```
-
-#### Advanced Querying
-```js
-// Complete example: Complex query with multiple conditions
-const connection = await Fliplet.DataSources.connectByName("Users");
-const seniorEngineers = await connection.find({
- where: {
- Office: 'London',
- Age: { $gte: 25 },
- Department: { $in: ['Engineering', 'Design'] },
- Status: 'Active'
- },
- order: [['data.Name', 'ASC']],
- limit: 50
-});
-console.log('Senior engineers in London:', seniorEngineers);
-
-// Complete example: Using Fliplet's custom $filters (optimized for performance)
-const connection = await Fliplet.DataSources.connectByName("Users");
-const activeUsers = await connection.find({
- where: {
- $filters: [
- {
- column: 'Email',
- condition: 'contains',
- value: '@company.com'
- },
- {
- column: 'Status',
- condition: '==',
- value: 'Active'
- }
- ]
- }
-});
-console.log('Active company users:', activeUsers);
-```
-
-> **Combining data from another data source?** Don't fetch a second data source and match records in JavaScript. Pass a `join` option to `find()` and the server returns the related records in a single query — see [Joining data from other data sources](#joining-data-from-other-data-sources) below and the full [joins documentation](datasources/joins).
-
-#### Query Operators Reference
-
-**MongoDB-style Operators (Sift.js)**
-```js
-// Comparison operators
-{ field: { $gt: 10 } } // Greater than
-{ field: { $gte: 10 } } // Greater than or equal
-{ field: { $lt: 10 } } // Less than
-{ field: { $lte: 10 } } // Less than or equal
-{ field: { $eq: 'value' } } // Equal to
-{ field: { $ne: 'value' } } // Not equal to
-
-// Array operators
-{ field: { $in: ['a', 'b'] } } // Value in array
-{ field: { $nin: ['a', 'b'] } } // Value not in array
-
-// Text operators
-{ field: { $iLike: 'john' } } // Case-insensitive partial match
-{ field: { $regex: /pattern/i } } // Regular expression
-
-// Logical operators
-{ $and: [{ field1: 'a' }, { field2: 'b' }] }
-{ $or: [{ field1: 'a' }, { field2: 'b' }] }
-{ $nor: [{ field1: 'a' }, { field2: 'b' }] } // None of the conditions
-{ field: { $not: { $lt: 18 } } } // Logical NOT
-
-// Other operators
-{ field: { $exists: true } } // Field exists
-{ field: { $mod: [4, 0] } } // Modulo operation
-{ field: { $all: ['a', 'b'] } } // Array contains all values
-{ field: { $size: 3 } } // Array size
-{ field: { $type: 'string' } } // Type check
-{ field: { $elemMatch: { $gte: 80 } } } // Array element matching
-```
-
-**Fliplet Custom $filters Operator**
-
-The `$filters` operator provides optimized performance and additional conditions:
-
-```js
-{
- where: {
- $filters: [
- // Exact match (case-insensitive)
- {
- column: 'Email',
- condition: '==',
- value: 'user@email.com'
- },
- // Not equal
- {
- column: 'Status',
- condition: '!=',
- value: 'Inactive'
- },
- // Numeric comparisons
- {
- column: 'Age',
- condition: '>', // '>', '>=', '<', '<='
- value: 25
- },
- // Contains (case-insensitive partial match)
- {
- column: 'Name',
- condition: 'contains',
- value: 'John'
- },
- // Empty/not empty checks
- {
- column: 'Notes',
- condition: 'empty' // or 'notempty'
- },
- // Range check
- {
- column: 'Score',
- condition: 'between',
- value: { from: 80, to: 100 }
- },
- // One of multiple values
- {
- column: 'Department',
- condition: 'oneof',
- value: ['Engineering', 'Design', 'Marketing']
- },
- // Date comparisons
- {
- column: 'Birthday',
- condition: 'dateis', // 'datebefore', 'dateafter', 'datebetween'
- value: '1990-01-01'
- }
- ]
- }
-}
-```
-
-**📖 Complete Operators Reference:** [View detailed query operators documentation](datasources/query-operators)
-
-### 2. Insert Records (Add Data)
-
-**Purpose:** Add new records to the data source
-**Syntax:** `connection.insert(data, options?)`
-**Returns:** `Promise`
-**When to use:** Creating new entries
-
-```js
-// Complete example: Insert a single user record
-const connection = await Fliplet.DataSources.connectByName("Users");
-const newUser = await connection.insert({
- Name: 'John Doe',
- Email: 'john.doe@company.com',
- Department: 'Engineering',
- Office: 'London',
- Age: 28,
- Status: 'Active'
-});
-console.log('Created new user:', newUser);
-
-// Complete example: Insert with acknowledgment for immediate local update
-const connection = await Fliplet.DataSources.connectByName("Users");
-const newUser = await connection.insert({
- Name: 'Jane Smith',
- Email: 'jane.smith@company.com',
- Department: 'Design',
- Office: 'New York',
- Age: 26,
- Status: 'Active'
-}, {
- ack: true // Ensure local database updates immediately
-});
-console.log('Created user with immediate sync:', newUser);
-
-// Complete example: Insert with file upload (FormData)
-const connection = await Fliplet.DataSources.connectByName("Users");
-const formData = new FormData();
-formData.append('Name', 'Bob Johnson');
-formData.append('Email', 'bob.johnson@company.com');
-formData.append('Department', 'Marketing');
-formData.append('Avatar', fileInput.files[0]); // File from input element
-
-const newUserWithFile = await connection.insert(formData, {
- folderId: 123 // Specify media folder for file uploads
-});
-console.log('Created user with avatar:', newUserWithFile);
-```
-
-**Options:**
-- `folderId` (Number): MediaFolder ID where uploaded files should be stored
-- `ack` (Boolean): If true, ensures the local offline database gets updated immediately
-
-### 3. Update Records (Modify Data)
-
-**Purpose:** Modify existing records
-**Syntax:** `connection.update(recordId, data, options?)`
-**Returns:** `Promise`
-**When to use:** Changing existing entries
-
-```js
-// Complete example: Update a user record by ID
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-// First find the user to get their ID
-const users = await connection.find({
- where: { Email: 'john.doe@company.com' }
-});
-
-if (users.length > 0) {
- const updatedUser = await connection.update(users[0].id, {
- Office: 'Berlin',
- Department: 'Product Engineering'
- });
- console.log('Updated user:', updatedUser);
-}
-
-// Complete example: Update user with file upload
-const connection = await Fliplet.DataSources.connectByName("Users");
-const formData = new FormData();
-formData.append('Office', 'San Francisco');
-formData.append('Avatar', newAvatarFile);
-
-const updatedUser = await connection.update(456, formData, {
- folderId: 789
-});
-console.log('Updated user with new avatar:', updatedUser);
-```
-
-### 4. Find Single Record
-
-**Purpose:** Get one specific record
-**Syntax:** `connection.findOne(options)` or `connection.findById(id)`
-**Returns:** `Promise`
-**When to use:** Looking for a specific entry
-
-```js
-// Complete example: Find one user by email
-const connection = await Fliplet.DataSources.connectByName("Users");
-const user = await connection.findOne({
- where: { Email: 'john.doe@company.com' }
-});
-
-if (user) {
- console.log('Found user:', user);
-} else {
- console.log('User not found');
-}
-
-// Complete example: Find user by ID (if you know the ID)
-const connection = await Fliplet.DataSources.connectByName("Users");
-const user = await connection.findById(123);
-
-if (user) {
- console.log('Found user by ID:', user);
-} else {
- console.log('User with ID 123 not found');
-}
-```
-
-### 5. Remove Records (Delete Data)
-
-**Purpose:** Delete records from the data source
-**Syntax:** `connection.removeById(id)` or `connection.query({ type: 'delete', where: {...} })`
-**Returns:** `Promise`
-**When to use:** Removing unwanted entries
-
-```js
-// Complete example: Remove user by ID
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-// First find the user to get their ID
-const users = await connection.find({
- where: { Email: 'obsolete.user@company.com' }
-});
-
-if (users.length > 0) {
- await connection.removeById(users[0].id);
- console.log('User removed successfully');
-}
-
-// Complete example: Remove multiple users matching criteria
-const connection = await Fliplet.DataSources.connectByName("Users");
-const deletedEntries = await connection.query({
- type: 'delete',
- where: { Status: 'Inactive' }
-});
-console.log('Removed inactive users:', deletedEntries);
-```
-
----
-
-## Bulk Operations
-
-### Insert Multiple Records
-
-**Purpose:** Add many records at once (more efficient than individual inserts)
-**Syntax:** `connection.append(recordsArray, options?)`
-**Returns:** `Promise`
-**When to use:** Bulk data import
-
-```js
-// Complete example: Add multiple users at once
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-const newUsers = [
- {
- Name: 'Alice Cooper',
- Email: 'alice.cooper@company.com',
- Department: 'Engineering',
- Office: 'London',
- Age: 29,
- Status: 'Active'
- },
- {
- Name: 'Bob Wilson',
- Email: 'bob.wilson@company.com',
- Department: 'Design',
- Office: 'New York',
- Age: 31,
- Status: 'Active'
- },
- {
- Name: 'Charlie Brown',
- Email: 'charlie.brown@company.com',
- Department: 'Marketing',
- Office: 'Berlin',
- Age: 27,
- Status: 'Active'
- }
-];
-
-await connection.append(newUsers);
-console.log(`Added ${newUsers.length} new users`);
-
-// Complete example: Bulk insert with disabled hooks for better performance
-const connection = await Fliplet.DataSources.connectByName("Users");
-await connection.append(newUsers, { runHooks: false });
-console.log('Bulk insert completed (hooks disabled for performance)');
-```
-
-### Replace All Data
-
-**Purpose:** Replace entire data source contents with new records
-**Syntax:** `connection.replaceWith(recordsArray)`
-**Returns:** `Promise`
-**When to use:** Complete data refresh
-
-```js
-// Complete example: Replace all users with new dataset
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-const newUserDataset = [
- {
- Name: 'Admin User',
- Email: 'admin@company.com',
- Department: 'IT',
- Office: 'London',
- Age: 35,
- Status: 'Active'
- },
- {
- Name: 'Test User',
- Email: 'test@company.com',
- Department: 'QA',
- Office: 'London',
- Age: 28,
- Status: 'Active'
- }
-];
-
-await connection.replaceWith(newUserDataset);
-console.log('Replaced all user data with new dataset');
-```
-
-### Commit Multiple Changes
-
-**Purpose:** Insert, update, and delete in a single operation
-**Syntax:** `connection.commit(options)`
-**Returns:** `Promise`
-**When to use:** Complex bulk operations
-
-```js
-// Complete example: Multiple operations in one transaction
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-await connection.commit({
- entries: [
- // Insert new user
- {
- data: {
- Name: 'New Employee',
- Email: 'new.employee@company.com',
- Department: 'Sales',
- Office: 'Paris',
- Age: 24,
- Status: 'Active'
- }
- },
-
- // Update existing user (assuming ID 123 exists)
- {
- id: 123,
- data: {
- Office: 'Remote',
- Department: 'Engineering - Remote'
- }
- }
- ],
-
- // Delete users by ID (assuming these IDs exist)
- delete: [456, 789],
-
- // Keep existing entries not mentioned
- append: true,
-
- // Merge with existing data instead of replacing
- extend: true,
-
- // Don't return all entries (faster)
- returnEntries: false
-});
-
-console.log('Bulk operations completed successfully');
-```
-
----
-
-## Pagination and Performance
-
-### Basic Pagination
-
-```js
-// Complete example: Paginated user retrieval
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-// Get first 10 users
-const page1 = await connection.find({
- limit: 10,
- offset: 0,
- order: [['data.Name', 'ASC']]
-});
-console.log('Page 1 users:', page1);
-
-// Get next 10 users
-const page2 = await connection.find({
- limit: 10,
- offset: 10,
- order: [['data.Name', 'ASC']]
-});
-console.log('Page 2 users:', page2);
-
-// Complete example: Get pagination info
-const connection = await Fliplet.DataSources.connectByName("Users");
-const result = await connection.find({
- limit: 10,
- offset: 0,
- includePagination: true,
- order: [['data.Name', 'ASC']]
-});
-
-console.log(`Users: ${result.entries.length}`);
-console.log(`Total users: ${result.pagination.total}`);
-console.log(`Page: ${(result.pagination.offset / result.pagination.limit) + 1}`);
-```
-
-### Advanced Pagination with Cursor
-
-**Purpose:** Navigate through large datasets efficiently
-**Syntax:** `connection.findWithCursor(options)`
-**Returns:** `Promise`
-**When to use:** Building paginated interfaces
-
-```js
-// Complete example: Using cursor for pagination
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-const cursor = await connection.findWithCursor({
- where: { Status: 'Active' },
- limit: 5,
- order: [['data.Name', 'ASC']]
-});
-
-console.log(`Page ${cursor.currentPage + 1} users:`, cursor);
-console.log(`Total users on page: ${cursor.length}`);
-console.log(`Is first page: ${cursor.isFirstPage}`);
-console.log(`Is last page: ${cursor.isLastPage}`);
-
-// Navigate to next page
-if (!cursor.isLastPage) {
- const nextPage = await cursor.next().update();
- console.log('Next page users:', nextPage);
-}
-
-// Navigate to previous page
-if (!cursor.isFirstPage) {
- const prevPage = await cursor.prev().update();
- console.log('Previous page users:', prevPage);
-}
-
-// Jump to specific page
-const page3 = await cursor.setPage(2).update(); // 0-indexed, so 2 = page 3
-console.log('Page 3 users:', page3);
-```
-
-**Cursor Properties:**
-- `limit`: The page size (number of records per page)
-- `offset`: Current offset from first page
-- `currentPage`: Current page number (0-based)
-- `isFirstPage`: Whether current page is first
-- `isLastPage`: Whether current page is last
-- `query`: Options used for the cursor
-
-**Cursor Methods:**
-- `next()`: Move to next page
-- `prev()`: Move to previous page
-- `setPage(n)`: Go to specific page
-- `update()`: Refresh current page data
-
----
-
-## Sorting and Ordering
-
-**Purpose:** Control the order of returned records
-**Syntax:** Use `order` array in find options
-**When to use:** When you need specific sorting
-
-### The `order` contract
-
-`order` is validated by the API before the query runs. A value that breaks the
-shape or column rules below is rejected with `400 Bad Request` — the query does
-not fall back to the default sorting.
-
-| Rule | What it means |
-|---|---|
-| **Shape** — `order` is an array of arrays | Each sort is its own `[column, direction]` pair, so the value is `order: [['data.Name', 'ASC']]`. A flat `order: ['data.Name', 'ASC']` is rejected. |
-| **Column** — five columns may appear unprefixed, and must be | Only `id`, `order`, `createdAt`, `deletedAt`, `updatedAt` are accepted without a prefix, and prefixing one names a column the data source does not declare — write `createdAt`, never `data.createdAt`. |
-| **Column** — every other column needs the `data.` prefix | Anything you added to the data source is an entry column, so it is written `data.` — `data.Name`, not `Name`. |
-| **Column** — the name after `data.` is verbatim | It must reproduce the declared column exactly, **spaces included** — `data.Start Time UTC`. Never replace a space with a dot, camel-case it, or truncate at it. |
-| **Direction** — write `'ASC'` or `'DESC'` | A convention, not a validated rule: any other value fails as a database error — still returned as a `400`, but carrying the database's message instead of the `Invalid order column name` validation error. Always write the direction as a literal in your code; never build it from user input. Omitting the direction defaults to `'ASC'`. |
-
-Flat arrays fail with a confusing error. A flat order: ['createdAt', 'DESC'] makes the API read the string one character at a time, so the response is Invalid order column name: c — a single letter that appears nowhere in your request. A one-character column name in the error almost always means the shape is flat and needs an extra pair of brackets. The remaining ways to produce one are rare: an unprefixed one-character column name such as order: [['N', 'ASC']], which is rejected for the missing data. prefix rather than for its shape, or a prefixed one-character name that is not a letter, digit, space, underscore or hyphen, such as order: [['data.@', 'ASC']].
-
-Never add the data. prefix to where keys or attributes entries. Both are applied inside the entry's data object, so they take entry column names unprefixed — where: { Name: 'John' } and attributes: ['Name', 'Email'] are correct, and adding data. to them breaks the query. Options that address the raw entry row instead do use the prefix, just like order: the keys and values of a join's on mapping (on: { 'data.CustomerEmail': 'data.Email' }) and the field paths in an aggregate pipeline ('$data.Department'). Omitting the prefix there raises no error — the join silently matches nothing (every record comes back with an empty joins entry, or drops out of the results entirely on a required join) and the aggregation reads undefined values.
-
-Confirm the column names against the data source's declared columns before sorting. On a data source that declares its columns, an entry column it does not declare fails with Column '<name>' does not exists, and a top-level column that is not declared (such as data.createdAt) fails the same way — createdAt belongs in the unprefixed list, not behind the prefix. On a data source with no declared columns the check is skipped, so an unknown column sorts silently on NULLs instead of erroring.
-
-Sort by creation date, newest first:
-
-```js
-const connection = await Fliplet.DataSources.connectByName("Users");
-const recentUsers = await connection.find({
- order: [['createdAt', 'DESC']], // 'createdAt' is one of the five unprefixed columns
- limit: 10
-});
-console.log('Most recent users:', recentUsers);
-```
-
-Sort by an entry column:
-
-```js
-const connection = await Fliplet.DataSources.connectByName("Users");
-const usersByName = await connection.find({
- order: [['data.Name', 'ASC']] // 'Name' is an entry column, so it needs the 'data.' prefix
-});
-console.log('Users sorted by name:', usersByName);
-```
-
-Sort by a column whose name contains spaces:
-
-```js
-const connection = await Fliplet.DataSources.connectByName("Sessions");
-const sessions = await connection.find({
- // The data source declares a column called: Start Time UTC
- order: [['data.Start Time UTC', 'DESC']] // spaces are kept exactly as declared
-});
-console.log('Most recent sessions:', sessions);
-```
-
-Sort by multiple columns:
-
-```js
-const connection = await Fliplet.DataSources.connectByName("Users");
-const sortedUsers = await connection.find({
- order: [
- ['data.Department', 'ASC'], // Sort by department first
- ['data.Name', 'ASC'] // Then by name
- ]
-});
-console.log('Users sorted by department then name:', sortedUsers);
-
-// Available sort columns:
-// - Fliplet columns: 'id', 'order', 'createdAt', 'deletedAt', 'updatedAt'
-// - Entry columns: 'data.ColumnName'
-// - Sort directions: 'ASC' (ascending), 'DESC' (descending)
-```
-
-### Common mistakes
-
-```js
-// WRONG: flat array — rejected with "Invalid order column name: c"
-order: ['createdAt', 'DESC']
-// RIGHT
-order: [['createdAt', 'DESC']]
-
-// WRONG: entry column without the prefix — rejected with "Invalid order column name: Name"
-order: [['Name', 'ASC']]
-// RIGHT
-order: [['data.Name', 'ASC']]
-
-// WRONG: space rewritten as a dot — the API reads the column as "Start"
-order: [['data.Start.Time.UTC', 'DESC']]
-// WRONG: space truncated — the API reads the column as "Start"
-order: [['data.Start', 'DESC']]
-// RIGHT: the declared column name, verbatim
-order: [['data.Start Time UTC', 'DESC']]
-```
-
----
+Connection and network methods return Promises. Subscription registration and cursor navigation return synchronous handles; their references identify which calls need `await`. Use the plural namespace `Fliplet.DataSources`.
-## Subscribing to Data Source Changes
+## Read records
-**Purpose:** Keep a screen up to date when other people change the data
-**Syntax:** `connection.subscribe(options, callback)`
-**Returns:** `Subscription`
-**When to use:** Screens that stay open while the data changes underneath them
-
-### How updates arrive
-
-Changes reach the screen within about 30 seconds rather than the instant they
-happen. Fliplet checks the data source for you on a repeating timer and calls
-your callback with whatever changed since the last check.
-
-Design around that delay. If something has to appear the moment it is created,
-a subscription is not the right tool.
-
-### What a subscription costs your app
-
-Every screen that subscribes does its own checking, and keeps checking for as
-long as it stays open:
-
-- **It adds up with people, not with data.** 200 people with the screen open
- means the data source is read around 400 times a minute, whether or not
- anything has changed.
-- **An empty data source costs the same as a busy one.** The check still runs;
- there is simply nothing to report.
-- **It carries on in the background.** A screen left open in a background tab
- keeps checking overnight, on every device that left it open.
-
-Those reads count toward your app's data source usage, so a subscription left
-running unnecessarily is one of the easier ways to build up usage without
-noticing it.
-
-The two rules below keep that under control. Follow both on any screen that
-stays open.
-
-### 1. Pause while the screen is hidden
-
-A subscription keeps running when someone switches to another app or tab.
-Cleaning up when the screen is destroyed is not enough on its own — people
-move away from a screen far more often than they close it, so without this the
-checks continue all night for everyone who left it open.
+This example needs a source named `Directory` with `Name` and `Department` columns and permission to query it. It uses an online connection and handles failed requests.
```js
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-const subscription = connection.subscribe({
- events: ['insert', 'update', 'delete']
-}, (changes) => {
- if (changes.inserted && changes.inserted.length) {
- // Add the new rows to your UI
- }
-
- if (changes.updated && changes.updated.length) {
- // Refresh the rows that changed
- }
-
- if (changes.deleted && changes.deleted.length) {
- // Remove the rows that went away
- }
-});
-
-function onVisibilityChange() {
- if (document.hidden) {
- if (subscription.status() === 'active') {
- subscription.pause();
- }
- } else if (subscription.status() === 'paused') {
- subscription.resume();
- }
-}
-
-document.addEventListener('visibilitychange', onVisibilityChange);
-
-// On teardown, remove the listener AND end the subscription.
-document.removeEventListener('visibilitychange', onVisibilityChange);
-subscription.unsubscribe();
-```
-
-`pause()` stops the checking, so nothing is read while the screen is hidden.
-`resume()` catches up straight away, so anything that changed in the meantime
-still reaches the screen. Check `status()` before switching, as shown above:
-calling `resume()` on a subscription that is already running has no effect.
-
-### 2. Never add a timer of your own on top
-
-Adding a `setInterval` that calls `find()` as a safety net, in case the
-subscription misses something, is the most common mistake here. The
-subscription already re-reads the data source on every check, so a second
-timer repeats work that has just been done and doubles what the screen costs
-without covering anything extra.
-
-If updates are not coming through, the cause is in your callback or in the
-`events` you asked for. A second timer hides that rather than fixing it.
-
----
-
-## Utility Methods
-
-### Get Unique Values
-
-**Purpose:** Get distinct values from columns
-**When to use:** Building filters, dropdowns, or analytics
-
-```js
-// Complete example: Get unique office locations
-const connection = await Fliplet.DataSources.connectByName("Users");
-const offices = await connection.getIndex('Office');
-console.log('Available offices:', offices);
-// Returns: ['London', 'New York', 'Berlin', 'Paris']
-
-// Complete example: Get unique values for multiple columns
-const connection = await Fliplet.DataSources.connectByName("Users");
-const indexes = await connection.getIndexes(['Office', 'Department']);
-console.log('Unique values:', indexes);
-// Returns: {
-// Office: ['London', 'New York', 'Berlin'],
-// Department: ['Engineering', 'Design', 'Marketing']
-// }
-
-// Use for building dynamic filters
-const { Office: availableOffices, Department: availableDepartments } = indexes;
-console.log('Build office filter with:', availableOffices);
-console.log('Build department filter with:', availableDepartments);
-```
-
-### Import Data from File
-
-**Purpose:** Import data from CSV or other file formats
-**Syntax:** `connection.import(FormData)`
-**Returns:** `Promise`
-**When to use:** Bulk data import from files
-
-```js
-// Complete example: Import users from CSV file
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-// Assuming you have a file input element
-const fileInput = document.getElementById('csvFileInput');
-const csvFile = fileInput.files[0];
-
-if (csvFile) {
- const formData = new FormData();
- formData.append('file', csvFile);
-
+async function readDirectory() {
try {
- await connection.import(formData);
- console.log('User data imported successfully from CSV');
- } catch (error) {
- console.error('Import failed:', error);
- }
-} else {
- console.log('No file selected');
-}
-```
-
----
-
-## Data Source Management
-
-### Get Available Data Sources
-
-**Purpose:** List data sources you can work with
-**Syntax:** `Fliplet.DataSources.get(options?)`
-**Returns:** `Promise`
-**When to use:** Discovery, building dynamic interfaces
-
-```js
-// Complete example: Get all data sources for organization
-const dataSources = await Fliplet.DataSources.get({
- attributes: ['id', 'name', 'columns']
-});
-console.log('Available data sources:', dataSources);
-
-// Find the Users data source
-const usersDataSource = dataSources.find(ds => ds.name === 'Users');
-if (usersDataSource) {
- console.log('Users data source found:', usersDataSource);
- console.log('Available columns:', usersDataSource.columns);
-}
-
-// Complete example: Get data sources used by current app
-const appDataSources = await Fliplet.DataSources.get({
- appId: Fliplet.Env.get('masterAppId'),
- includeInUse: true
-});
-console.log('App data sources:', appDataSources);
-
-// Complete example: Get specific data source details by ID
-const dataSource = await Fliplet.DataSources.getById(123, {
- attributes: ['name', 'hooks', 'columns']
-});
-console.log('Data source details:', dataSource);
-```
-
-### Create New Data Source
-
-**Purpose:** Programmatically create data sources
-**Syntax:** `Fliplet.DataSources.create(options)`
-**Returns:** `Promise`
-**When to use:** Dynamic app setup, automated workflows
-
-```js
-// Complete example: Create a new Users data source
-const newDataSource = await Fliplet.DataSources.create({
- name: 'Users',
-
- // Attach to current app
- appId: Fliplet.Env.get('appId'),
- organizationId: Fliplet.Env.get('organizationId'),
-
- // Define structure
- columns: ['Name', 'Email', 'Department', 'Office', 'Age', 'Status'],
-
- // Add initial test data
- entries: [
- {
- Name: 'John Smith',
- Email: 'john.smith@company.com',
- Department: 'Engineering',
- Office: 'London',
- Age: 30,
- Status: 'Active'
- },
- {
- Name: 'Sarah Jones',
- Email: 'sarah.jones@company.com',
- Department: 'Design',
- Office: 'New York',
- Age: 28,
- Status: 'Active'
- }
- ],
-
- // Set permissions
- accessRules: [
- { type: ['select', 'insert', 'update', 'delete'], allow: 'all' }
- ]
-});
-
-console.log('Created Users data source:', newDataSource);
-
-// Now you can connect to it by name
-const connection = await Fliplet.DataSources.connectByName("Users");
-console.log('Connected to new Users data source');
-```
-
----
-
-## Advanced Features
-
-### Aggregation Queries
-
-**Purpose:** Run complex data analysis using MongoDB-style aggregation
-**Syntax:** Use `aggregate` option in find method
-**When to use:** Analytics, reporting, data transformation
-
-```js
-// Complete example: Group users by department and calculate average age
-const connection = await Fliplet.DataSources.connectByName("Users");
-
-const departmentStats = await connection.find({
- aggregate: [
- {
- $project: {
- department: '$data.Department',
- numericAge: { $convertToNumber: '$data.Age' }
- }
- },
- {
- $group: {
- _id: '$department',
- avgAge: { $avg: '$numericAge' },
- userCount: { $sum: 1 }
- }
- }
- ]
-});
-
-console.log('Department statistics:', departmentStats);
-// Example output: [
-// { _id: 'Engineering', avgAge: 29.5, userCount: 4 },
-// { _id: 'Design', avgAge: 27.2, userCount: 3 }
-// ]
-```
-
-**Note:** Use the custom `$convertToNumber` operator to convert strings to numbers before aggregation.
-
-### Joining Data from Other Data Sources
-
-When a screen shows records from one data source enriched with fields from another (orders with their customer's name/email, articles with their comments), use the **native server-side join** — pass a `join` option to `find()`. Keep the schema normalized: store a reference key on the child record (e.g. `CustomerEmail` on an order), not a duplicated copy of the other record's fields.
-
-```js
-// Orders, each carrying its customer's record from a separate Customers data source
-const orders = await Fliplet.DataSources.connect(611);
-const result = await orders.find({
- join: {
- Customer: {
- dataSourceId: 610, // the related (Customers) data source
- on: { 'data.CustomerEmail': 'data.Email' } // childKey: parentKey
- }
- }
-});
-// each row now carries result[i].joins.Customer with the joined customer record
-// (the joined data sits alongside result[i].data, not inside it)
-```
-
-This is one query the server resolves — **don't** fetch every record from the second data source and match them in JavaScript; that doesn't paginate or scale. For join types (left/inner), the `on` mapping, and filtering joined data, see the full [joins documentation](datasources/joins).
-
-### Data Source Views
-
-For filtering data sources with predefined views, see the [views documentation](datasources/views).
-
----
-
-## Common Usage Patterns
-
-### Pattern 1: Complete User Management System
-```js
-// Initialize user management with connection
-const initUserManager = async () => {
- const connection = await Fliplet.DataSources.connectByName("Users");
- console.log('UserManager initialized');
- return connection;
-};
-
-// Create new user
-const createUser = async (connection, { name, email, department, office, age }) => {
- const newUser = await connection.insert({
- Name: name,
- Email: email,
- Department: department,
- Office: office,
- Age: age,
- Status: 'Active',
- CreatedDate: new Date().toISOString().split('T')[0]
- });
-
- console.log(`Created user: ${newUser.data.Name}`);
- return newUser;
-};
-
-// Get all active users
-const getActiveUsers = async (connection) => {
- const users = await connection.find({
- where: { Status: 'Active' },
- order: [['data.Name', 'ASC']]
- });
-
- console.log(`Found ${users.length} active users`);
- return users;
-};
-
-// Update user information
-const updateUser = async (connection, email, updates) => {
- const users = await connection.find({
- where: { Email: email }
- });
-
- if (users.length > 0) {
- const updatedUser = await connection.update(users[0].id, updates);
- console.log(`Updated user: ${email}`);
- return updatedUser;
- } else {
- throw new Error(`User with email ${email} not found`);
- }
-};
-
-// Deactivate user (soft delete)
-const deactivateUser = async (connection, email) => {
- return updateUser(connection, email, { Status: 'Inactive' });
-};
-
-// Usage example
-const manageUsers = async () => {
- const connection = await initUserManager();
-
- // Create a new user
- const newUser = await createUser(connection, {
- name: 'Alice Johnson',
- email: 'alice.johnson@company.com',
- department: 'Marketing',
- office: 'Berlin',
- age: 26
- });
-
- // Get all active users
- const activeUsers = await getActiveUsers(connection);
-
- // Update user
- await updateUser(connection, 'alice.johnson@company.com', {
- Office: 'Remote',
- Department: 'Digital Marketing'
- });
-
- // Deactivate user
- await deactivateUser(connection, 'alice.johnson@company.com');
-};
-
-// Run the user management example
-manageUsers().catch(console.error);
-```
-
-### Pattern 2: Advanced Search and Filtering
-```js
-// Initialize search functionality
-const initUserSearch = async () => {
- const connection = await Fliplet.DataSources.connectByName("Users");
- console.log('User search initialized');
- return connection;
-};
-
-// Search users with multiple criteria
-const searchUsers = async (connection, {
- searchTerm = '',
- departments = [],
- offices = [],
- minAge = 0,
- maxAge = 100,
- status = 'Active'
-} = {}) => {
-
- const whereConditions = {
- $and: [
- { Status: status },
- { Age: { $gte: minAge, $lte: maxAge } }
- ]
- };
-
- // Add text search if provided
- if (searchTerm) {
- whereConditions.$and.push({
- $or: [
- { Name: { $iLike: searchTerm } },
- { Email: { $iLike: searchTerm } }
- ]
+ const connection = await Fliplet.DataSources.connectByName('Directory', {
+ offline: false
});
- }
-
- // Add department filter if provided
- if (departments.length > 0) {
- whereConditions.$and.push({
- Department: { $in: departments }
- });
- }
-
- // Add office filter if provided
- if (offices.length > 0) {
- whereConditions.$and.push({
- Office: { $in: offices }
- });
- }
-
- const results = await connection.find({
- where: whereConditions,
- order: [['data.Name', 'ASC']],
- limit: 100
- });
-
- console.log(`Search found ${results.length} users`);
- return results;
-};
-
-// Get filter options for building UI
-const getFilterOptions = async (connection) => {
- const [departments, offices] = await Promise.all([
- connection.getIndex('Department'),
- connection.getIndex('Office')
- ]);
-
- return { departments, offices };
-};
-
-// Advanced search using $filters for better performance
-const searchUsersOptimized = async (connection, filters = {}) => {
- const {
- emailDomain = '@company.com',
- department,
- office,
- ageRange = { min: 18, max: 65 }
- } = filters;
-
- const results = await connection.find({
- where: {
- $filters: [
- {
- column: 'Status',
- condition: '==',
- value: 'Active'
- },
- {
- column: 'Email',
- condition: 'contains',
- value: emailDomain
- },
- {
- column: 'Age',
- condition: 'between',
- value: { from: ageRange.min, to: ageRange.max }
- },
- ...(department ? [{
- column: 'Department',
- condition: '==',
- value: department
- }] : []),
- ...(office ? [{
- column: 'Office',
- condition: '==',
- value: office
- }] : [])
- ]
- }
- });
-
- return results;
-};
-
-// Usage example
-const demonstrateSearch = async () => {
- const connection = await initUserSearch();
-
- // Get filter options for UI
- const { departments, offices } = await getFilterOptions(connection);
- console.log('Available departments:', departments);
- console.log('Available offices:', offices);
-
- // Search with multiple criteria
- const searchResults = await searchUsers(connection, {
- searchTerm: 'john',
- departments: ['Engineering', 'Design'],
- offices: ['London', 'Berlin'],
- minAge: 25,
- maxAge: 40
- });
- console.log('Search results:', searchResults);
-
- // Optimized search
- const optimizedResults = await searchUsersOptimized(connection, {
- department: 'Engineering',
- office: 'London',
- ageRange: { min: 25, max: 35 }
- });
- console.log('Optimized search results:', optimizedResults);
-};
-
-// Run the search example
-demonstrateSearch().catch(console.error);
-```
-
-### Pattern 3: Paginated User Directory with Real-time Updates
-```js
-// Initialize paginated directory
-const initUserDirectory = async (pageSize = 10) => {
- const connection = await Fliplet.DataSources.connectByName("Users");
-
- const currentCursor = await connection.findWithCursor({
- where: { Status: 'Active' },
- limit: pageSize,
- order: [['data.Name', 'ASC']]
- });
-
- console.log('User directory initialized');
- return { connection, currentCursor };
-};
-
-// Display current page
-const displayCurrentPage = (cursor) => {
- const { currentPage, isFirstPage, isLastPage, length } = cursor;
-
- console.log(`\n=== User Directory - Page ${currentPage + 1} ===`);
- console.log(`Showing ${length} users:`);
-
- cursor.forEach((user, index) => {
- const { Name, Email, Department, Office } = user.data;
- console.log(`${index + 1}. ${Name} (${Email}) - ${Department}, ${Office}`);
- });
-
- console.log(`\nNavigation: First Page: ${isFirstPage}, Last Page: ${isLastPage}`);
-};
-
-// Navigate to next page
-const nextPage = async (cursor) => {
- if (!cursor.isLastPage) {
- const updatedCursor = await cursor.next().update();
- displayCurrentPage(updatedCursor);
- return updatedCursor;
- } else {
- console.log('Already on last page');
- return cursor;
- }
-};
-
-// Navigate to previous page
-const previousPage = async (cursor) => {
- if (!cursor.isFirstPage) {
- const updatedCursor = await cursor.prev().update();
- displayCurrentPage(updatedCursor);
- return updatedCursor;
- } else {
- console.log('Already on first page');
- return cursor;
- }
-};
-
-// Go to specific page
-const goToPage = async (cursor, pageNumber) => {
- const updatedCursor = await cursor.setPage(pageNumber - 1).update();
- displayCurrentPage(updatedCursor);
- return updatedCursor;
-};
-
-// Subscribe to updates.
-// This keeps checking for as long as the screen is open — see "Subscribing to
-// Data Source Changes" above. On a real screen, also pause it while the screen
-// is hidden; that is left out here to keep the pagination example readable.
-const subscribeToUpdates = (connection, cursor) => {
- const subscription = connection.subscribe({
- events: ['insert', 'update', 'delete']
- }, (changes) => {
- const { inserted, updated, deleted } = changes;
-
- if (inserted?.length) {
- console.log(`\n🆕 ${inserted.length} new user(s) added:`);
- inserted.forEach(user => {
- console.log(` + ${user.data.Name} (${user.data.Email})`);
- });
- }
-
- if (updated?.length) {
- console.log(`\n✏️ ${updated.length} user(s) updated:`);
- updated.forEach(user => {
- console.log(` ~ ${user.data.Name} (${user.data.Email})`);
- });
- }
-
- if (deleted?.length) {
- console.log(`\n🗑️ ${deleted.length} user(s) deleted`);
- }
-
- // Refresh current page to show changes
- refreshCurrentPage(cursor);
- });
-
- return subscription;
-};
-
-// Refresh current page
-const refreshCurrentPage = async (cursor) => {
- const updatedCursor = await cursor.update();
- console.log('\n--- Refreshed current page ---');
- displayCurrentPage(updatedCursor);
- return updatedCursor;
-};
-
-// Cleanup function
-const cleanup = (subscription) => {
- if (subscription) {
- subscription.unsubscribe();
- console.log('Real-time updates stopped');
- }
-};
-
-// Usage example
-const demonstrateDirectory = async () => {
- const { connection, currentCursor } = await initUserDirectory(5); // 5 users per page
- let cursor = currentCursor;
-
- // Display initial page
- displayCurrentPage(cursor);
-
- // Subscribe to updates
- const subscription = subscribeToUpdates(connection, cursor);
-
- // Simulate navigation after delays
- setTimeout(async () => {
- cursor = await nextPage(cursor);
- }, 2000);
-
- setTimeout(async () => {
- cursor = await previousPage(cursor);
- }, 4000);
-
- setTimeout(async () => {
- cursor = await goToPage(cursor, 3);
- }, 6000);
-
- // Cleanup after 30 seconds
- setTimeout(() => {
- cleanup(subscription);
- }, 30000);
-};
-
-// Run the directory example
-demonstrateDirectory().catch(console.error);
-```
-
-### Pattern 4: Comprehensive Error Handling and Recovery
-```js
-// Utility function for delays
-const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
-
-// Connect with retry logic
-const connectWithRetry = async (maxRetries = 3, retryDelay = 1000) => {
- for (let attempt = 1; attempt <= maxRetries; attempt++) {
- try {
- return await Fliplet.DataSources.connectByName("Users");
- } catch (error) {
- console.warn(`Connection attempt ${attempt} failed:`, error.message);
-
- if (attempt === maxRetries) {
- throw new Error(`Failed to connect after ${maxRetries} attempts`);
- }
-
- await delay(retryDelay * attempt);
- }
- }
-};
-
-// Safe operation wrapper with error handling
-const safeOperation = async (operation, operationName) => {
- try {
- const result = await operation();
- console.log(`✅ ${operationName} completed successfully`);
- return result;
+ const entries = await connection.find({
+ where: { Department: 'Operations' },
+ attributes: ['Name', 'Department'],
+ order: [['data.Name', 'ASC']],
+ limit: 25
+ }, { cache: false });
+ return entries.map(entry => ({ id: entry.id, name: entry.data.Name }));
} catch (error) {
- const { status, message } = error;
-
- console.error(`❌ ${operationName} failed:`, message);
-
- switch (status) {
- case 404:
- throw new Error(`${operationName} failed: Data source or record not found`);
- case 403:
- throw new Error(`${operationName} failed: Access denied. Check permissions.`);
- case 429:
- console.log('Rate limit hit, waiting before retry...');
- await delay(5000);
- return safeOperation(operation, operationName);
- case 500:
- throw new Error(`${operationName} failed: Server error. Please try again later.`);
- default:
- throw new Error(`${operationName} failed: ${message}`);
- }
- }
-};
-
-// Initialize robust user service
-const initRobustUserService = async () => {
- try {
- const connection = await connectWithRetry();
- console.log('✅ UserService initialized successfully');
- return connection;
- } catch (error) {
- console.error('❌ Failed to initialize UserService:', error.message);
+ console.error('Directory query failed', error);
throw error;
}
-};
-
-// Get user safely with error handling
-const getUserSafely = async (connection, email) => {
- return safeOperation(async () => {
- const users = await connection.find({
- where: { Email: email }
- });
-
- if (users.length === 0) {
- throw new Error(`User with email ${email} not found`);
- }
-
- return users[0];
- }, `Get user ${email}`);
-};
-
-// Create user safely with validation
-const createUserSafely = async (connection, userData) => {
- return safeOperation(async () => {
- // Validate required fields
- const requiredFields = ['Name', 'Email', 'Department'];
- const missingFields = requiredFields.filter(field => !userData[field]);
-
- if (missingFields.length > 0) {
- throw new Error(`Missing required fields: ${missingFields.join(', ')}`);
- }
-
- // Check if user already exists
- const existingUsers = await connection.find({
- where: { Email: userData.Email }
- });
-
- if (existingUsers.length > 0) {
- throw new Error(`User with email ${userData.Email} already exists`);
- }
-
- // Create user with defaults
- const userToCreate = {
- Status: 'Active',
- CreatedDate: new Date().toISOString().split('T')[0],
- ...userData
- };
-
- return await connection.insert(userToCreate);
- }, `Create user ${userData.Email}`);
-};
-
-// Update user safely with validation
-const updateUserSafely = async (connection, email, updates) => {
- return safeOperation(async () => {
- const user = await getUserSafely(connection, email);
-
- // Prevent email changes for data integrity
- if (updates.Email && updates.Email !== email) {
- throw new Error('Email address cannot be changed');
- }
-
- const updatedUser = await connection.update(user.id, {
- ...updates,
- LastModified: new Date().toISOString().split('T')[0]
- });
-
- return updatedUser;
- }, `Update user ${email}`);
-};
-
-// Get all users with fallback handling
-const getAllUsersWithFallback = async (connection) => {
- return safeOperation(async () => {
- const users = await connection.find({
- where: { Status: 'Active' },
- order: [['data.Name', 'ASC']]
- });
-
- if (users.length === 0) {
- console.warn('⚠️ No active users found');
- }
-
- return users;
- }, 'Get all users');
-};
-
-// Usage example with comprehensive error handling
-const demonstrateRobustOperations = async () => {
- try {
- const connection = await initRobustUserService();
-
- // Safe operations
- const newUser = await createUserSafely(connection, {
- Name: 'Jane Doe',
- Email: 'jane.doe@company.com',
- Department: 'Engineering',
- Office: 'London',
- Age: 29
- });
-
- const user = await getUserSafely(connection, 'jane.doe@company.com');
-
- const updatedUser = await updateUserSafely(connection, 'jane.doe@company.com', {
- Office: 'Remote',
- Age: 30
- });
-
- const allUsers = await getAllUsersWithFallback(connection);
-
- console.log('All operations completed successfully');
-
- } catch (error) {
- console.error('Application error:', error.message);
- // Handle error appropriately in your app
- }
-};
-
-// Run the robust operations example
-demonstrateRobustOperations().catch(console.error);
+}
```
----
-
-## Test Data Source Structure
-
-To test all the examples above, you can create a "Users" data source with the following structure:
-
-### Data Source: "Users"
+The default read result is an array of entries. Each entry has metadata such as `id`, `createdAt` and `updatedAt`, with column values under `data`:
-**Columns:**
-- `Name` (Text) - Full name of the user
-- `Email` (Text) - Email address (should be unique)
-- `Department` (Text) - Department name
-- `Office` (Text) - Office location
-- `Age` (Number) - Age in years
-- `Status` (Text) - Active/Inactive status
-
-**Sample Test Data:**
```json
-[
- {
- "Name": "John Smith",
- "Email": "john.smith@company.com",
- "Department": "Engineering",
- "Office": "London",
- "Age": 30,
- "Status": "Active"
- },
- {
- "Name": "Sarah Johnson",
- "Email": "sarah.johnson@company.com",
- "Department": "Design",
- "Office": "New York",
- "Age": 28,
- "Status": "Active"
- },
- {
- "Name": "Mike Wilson",
- "Email": "mike.wilson@company.com",
- "Department": "Engineering",
- "Office": "London",
- "Age": 32,
- "Status": "Active"
- },
- {
- "Name": "Emily Davis",
- "Email": "emily.davis@company.com",
- "Department": "Marketing",
- "Office": "Berlin",
- "Age": 26,
- "Status": "Active"
- },
- {
- "Name": "David Brown",
- "Email": "david.brown@company.com",
- "Department": "Sales",
- "Office": "Paris",
- "Age": 29,
- "Status": "Active"
- },
- {
- "Name": "Lisa Garcia",
- "Email": "lisa.garcia@company.com",
- "Department": "Design",
- "Office": "New York",
- "Age": 31,
- "Status": "Active"
- },
- {
- "Name": "Tom Anderson",
- "Email": "tom.anderson@company.com",
- "Department": "Engineering",
- "Office": "Berlin",
- "Age": 27,
- "Status": "Inactive"
- },
- {
- "Name": "Anna Martinez",
- "Email": "anna.martinez@company.com",
- "Department": "HR",
- "Office": "London",
- "Age": 33,
- "Status": "Active"
- },
- {
- "Name": "Chris Taylor",
- "Email": "chris.taylor@company.com",
- "Department": "Marketing",
- "Office": "Remote",
- "Age": 25,
- "Status": "Active"
- },
- {
- "Name": "Jennifer Lee",
- "Email": "jennifer.lee@company.com",
- "Department": "Finance",
- "Office": "New York",
- "Age": 34,
- "Status": "Active"
- }
-]
-```
-
-### Creating the Test Data Source
-
-You can create this data source programmatically using the API:
-
-```js
-// Create the Users data source with test data
-const testDataSource = await Fliplet.DataSources.create({
- name: 'Users',
- appId: Fliplet.Env.get('appId'),
- organizationId: Fliplet.Env.get('organizationId'),
- columns: ['Name', 'Email', 'Department', 'Office', 'Age', 'Status'],
- entries: [
- // ... paste the sample test data above
- ],
- accessRules: [
- { type: ['select', 'insert', 'update', 'delete'], allow: 'all' }
- ]
-});
-
-console.log('Test Users data source created:', testDataSource);
-```
-
-This test data provides:
-- **Variety**: Multiple departments, offices, and age ranges
-- **Real scenarios**: Active/inactive users, different locations
-- **Search testing**: Names and emails for text search testing
-- **Filter testing**: Multiple values for department and office filters
-- **Pagination testing**: 10 records for testing pagination features
-- **Age ranges**: Different ages for numeric filtering tests
-
-All examples in this documentation use this data structure and will work with this test data.
-
----
-
-## Offline database (native only)
-
-`Fliplet.DataSources.Database` is the offline-first database API for native Fliplet apps. It loads bundled JSON files from device storage on native (or fetches them via the REST bundle endpoint on web during development), then exposes a synchronous collection handle for CRUD and diff operations against the in-memory data.
-
-This API is primarily for native apps. On web it falls back to fetching the bundle, which is suitable for dev/test only.
-
-**Important notes:**
-
-- **Module-level cache.** Once a bundle is loaded the parsed object is reused for the process lifetime — subsequent calls to `Fliplet.DataSources.Database` with the same identifier do not re-read the file.
-- **No write batching.** Every mutating collection method (`insert`, `update`, `remove`, `applyDiff`) writes the full JSON file to disk via `Fliplet.Native.Maintenance.writeFile` within the same Promise chain.
-- **`update()` deep-merges.** Pass only the fields you want changed; existing fields not present in the second argument are left intact.
-- **`applyDiff({ fullRefresh: true })`** replaces the entire local collection with the server payload. Without `fullRefresh`, the diff removes entries by id and then appends the updated set.
-
-### Opening the database
-
-`Fliplet.DataSources.Database(options)` → `Promise`
-
-| Option | Type | Required | Description |
-|---|---|---|---|
-| `id` | Number \| String | Yes (unless `isName`) | Data source ID, or name when `isName: true` |
-| `isName` | Boolean | No | Set to `true` when `id` holds the data source name |
-| `name` | String | No | Alternative way to supply the name (used together with `isName: true`) |
-| `appId` | Number | No | Defaults to `Fliplet.Env.get('appId')` |
-
-```js
-// Open by data source ID
-const db = await Fliplet.DataSources.Database({ id: 42 });
-
-// Open by data source name
-const db = await Fliplet.DataSources.Database({ id: 'Products', isName: true });
-```
-
-### Selecting a data source
-
-`database.dataSource(id, opts?)` → `CollectionHandle` (synchronous — returns immediately, no Promise)
-
-```js
-const db = await Fliplet.DataSources.Database({ id: 42 });
-
-// Select by numeric ID
-const products = db.dataSource(42);
-
-// Select by name using the opts object
-const inventory = db.dataSource(null, { name: 'Inventory' });
-```
-
-### Reading data
-
-#### `find(where, options?)` → `Promise`
-
-Runs a [sift.js](https://github.com/crcn/sift.js) MongoDB-style query against `entry.data`. Supports data-source views (row-level filters) when `options.views` is provided.
-
-| Option | Type | Description |
-|---|---|---|
-| `limit` | Number | Maximum number of entries to return |
-| `views` | Array\ | Names of data-source views to apply as row-level filters |
-
-```js
-const db = await Fliplet.DataSources.Database({ id: 42 });
-const products = db.dataSource(42);
-
-// Return all entries
-const all = await products.find({});
-
-// Filter by category
-const electronics = await products.find({ Category: 'Electronics' });
-
-// MongoDB-style operators
-const affordable = await products.find(
- { Price: { $lte: 99.99 }, InStock: true },
- { limit: 20 }
-);
-
-// Apply a pre-configured view called "PublicCatalogue"
-const publicItems = await products.find({}, { views: ['PublicCatalogue'] });
-```
-
-#### `findById(id)` → `Promise`
-
-```js
-const product = await products.findById(101);
-
-if (product) {
- console.log(product.data.Name, product.data.Price);
+{
+ "id": 101,
+ "data": { "Name": "Ada Patel", "Department": "Operations" },
+ "createdAt": "2026-01-10T09:00:00.000Z",
+ "updatedAt": "2026-01-10T09:00:00.000Z"
}
```
-### Writing data
-
-All write methods mutate the in-memory collection first, then persist the full JSON bundle to disk.
-
-#### `insert(entry)` → `Promise`
-
-```js
-const db = await Fliplet.DataSources.Database({ id: 55 });
-const inventory = db.dataSource(55);
-
-const newItem = await inventory.insert({
- id: 9001,
- data: {
- SKU: 'WH-001',
- Name: 'Wireless Headphones',
- Price: 79.99,
- InStock: true,
- Category: 'Electronics'
- }
-});
-```
-
-#### `update(existingEntry, entry)` → `Promise`
-
-Deep-merges `entry` into `existingEntry` in-place. Pass only the fields you want changed.
-
-```js
-const db = await Fliplet.DataSources.Database({ id: 55 });
-const inventory = db.dataSource(55);
-
-const headphones = await inventory.findById(9001);
-
-// Update price and stock status only — all other fields are preserved
-await inventory.update(headphones, {
- data: { Price: 69.99, InStock: false }
-});
-```
-
-#### `remove(entry)` → `Promise`
-
-Accepts an entry object (matched by `id`) or a predicate function.
-
-```js
-const db = await Fliplet.DataSources.Database({ id: 55 });
-const inventory = db.dataSource(55);
-
-// Remove a specific entry by reference
-const item = await inventory.findById(9001);
-await inventory.remove(item);
-
-// Remove all out-of-stock entries using a predicate
-await inventory.remove(entry => entry.data.InStock === false);
-```
-
-### Applying server diffs
-
-`applyDiff(diff)` → `Promise`
+A query's `where` filters column values under `data`; entry metadata filters use top-level query parameters. Pagination, aggregation and request options can change the result shape. See [reading data](datasources/reading-data.md) before combining them.
-Used internally by the Fliplet native update pipeline to apply incremental server changes to the local bundle.
+## API navigation
-| Field | Type | Description |
-|---|---|---|
-| `updated` | Array | Entries to upsert (replace or append) |
-| `deleted` | Array | Entries to remove by id |
-| `fullRefresh` | Boolean | When `true`, replaces the entire local collection with `updated` |
+| Task | Reference and methods |
+|---|---|
+| Read, filter and paginate | [Reading data](datasources/reading-data.md): `find`, `query`, `findOne`, `findById`, `findWithCursor`, cursor methods, `getIndex`, `getIndexes` and aggregation |
+| Query operators | [Query operators](datasources/query-operators.md): JSON-safe filters and separate server rule lookup semantics |
+| Related records | [Joins](datasources/joins.md): named joins, reductions and pagination limits |
+| Session-bound filters | [Views](datasources/views.md): definitions, requested views and security boundaries |
+| Write records | [Writing data](datasources/writing-data.md): `insert`, `update`, `removeById`, query writes, `append`, `replaceWith` and `commit` |
+| Attach files or import | [File attachments](datasources/file-attachments.md): attachment inputs, limits and `import` capability boundaries |
+| Listen for changes | [Subscriptions](datasources/subscriptions.md): `subscribe` and subscription handle methods |
+| Native local storage | [Offline database](datasources/offline-database.md): `Database`, handles and collections |
+| Manage source settings | [Managing data sources](datasources/managing-data-sources.md): `get`, `getById`, `create`, namespace `update`, `delete` and source roles |
+| Authenticate app users | [App authentication](v3/auth.md): `sendValidation` and `validate` |
+| Secure and verify access | [Security entry](../Data-source-security.md), [rule reference](datasources/security-rules.md), [policy examples](datasources/security-examples.md) and [preview testing](datasources/testing-security.md) |
+| Server hooks | [Data source hooks](../Data-Source-Hooks.md) |
+
+## Connect to a source
+
+`Fliplet.DataSources.connect(id, connectionOptions?)` and `Fliplet.DataSources.connectByName(name, connectionOptions?)` return `Promise`. The options-object overload `connect({ name, offline })` also selects a named source.
+
+| Argument | Type and behavior |
+|---|---|
+| `id` | Required source identifier for the ID form. A missing/falsy identifier rejects. |
+| `name` | Required source name for the named form. Name lookup uses the current request context; it does not grant cross-app access. |
+| `connectionOptions.offline` | Optional Boolean. `false` forces online operations. When omitted, native apps can use bundled storage if the Database API is available. `true` does not enable native storage in a browser. |
```js
-const db = await Fliplet.DataSources.Database({ id: 42 });
-const products = db.dataSource(42);
-
-// Incremental update from server
-await products.applyDiff({
- updated: [
- { id: 101, data: { Name: 'Laptop Pro', Price: 1299.99, InStock: true } },
- { id: 102, data: { Name: 'USB-C Hub', Price: 49.99, InStock: true } }
- ],
- deleted: [
- { id: 88 }
- ]
-});
-
-// Full refresh — replace the entire local collection
-await products.applyDiff({
- fullRefresh: true,
- updated: [
- { id: 201, data: { Name: 'Monitor 4K', Price: 499.99, InStock: true } }
- ],
- deleted: []
-});
+const online = await Fliplet.DataSources.connectByName('Directory', { offline: false });
+const byId = await Fliplet.DataSources.connect(123, { offline: false });
```
-### Indexes
+The SDK may reuse a connection for the same identifier and offline setting. Connecting establishes a handle; it does not prove a successful server query or authorized access. Native bundle loading is lazy and failed local loading can cause later reads to use the server. There is no public connection `close`, `disconnect` or `destroy` method. Unsubscribe each listener when its owner is disposed.
-#### `indexes(columnsArray)` → `Promise`
+## Security and verification
-Returns unique values per column across all entries in the collection. If the bundle already contains a pre-computed `indexes` map, that is used directly; otherwise unique values are derived by scanning entries.
+Query reads and record reads evaluate different requirement targets. Object update requirements inspect stored rows, not proposed values. Field restrictions vary across single and bulk writes. Views, projections and client hooks do not replace server access rules.
-```js
-const db = await Fliplet.DataSources.Database({ id: 55 });
-const inventory = db.dataSource(55);
-
-const uniqueValues = await inventory.indexes(['Category', 'InStock']);
-// { Category: ['Electronics', 'Accessories'], InStock: [true, false] }
-```
+For authorization tests, use an online connection, the intended app identity and confirmed preview enforcement. Check denied operations, returned data and persisted effects. Administrative calls, local results and preview with enforcement disabled cannot establish app-user authorization. A resolved write may be queued or pending; use the write reference's persistence checks.
-#### `index(columnName)` → `Promise`
+## SDK helper boundaries
-Returns unique values for a single column.
+These exposed helpers have narrower purposes than record CRUD. They are not additional authorization or synchronization APIs.
-```js
-const categories = await inventory.index('Category');
-// ['Electronics', 'Accessories', 'Peripherals']
-```
-
----
-
-[Back to API documentation](../API-Documentation)
-{: .buttons}
+| Helper | Signature and outcome |
+|---|---|
+| Temporary IDs | `isTempId(id)` returns a Boolean synchronously; true for strings starting with `temp_`. A false result alone does not prove server persistence. |
+| Connection lookup | `clearActiveConnections()` returns `undefined` and resets the reuse lookup. It does not unsubscribe existing listeners. |
+| Client hook registration | `bindHooks(id, hooks)` registers an object synchronously and rejects duplicate bindings or nonobject inputs by throwing. These run in the client, separately from server hooks. |
+| Client hook lookup | `hasHook(id)` returns a Boolean synchronously. |
+| Client hook application | `applyHooks(id, method, entries)` invokes the bound callback. Array input returns `Promise.all` of callback results; a single item returns the callback's value/Promise. Without a callback it resolves `undefined`. |
+| Mapped-row convenience | `fetchWithOptions(options)` returns a Promise. See [mapped rows](datasources/managing-data-sources.md#mapped-row-helper) for its different result shape and options. |
+
+Underscore-prefixed tracking/reconciliation helpers and core queue-storage helpers are internal implementation surfaces. Use the documented insert/update outcomes rather than manipulating the queue. Native `applyDiff` belongs to the update pipeline, not a custom app synchronization recipe.
+
+## Existing reference links
+
+Existing method-family fragments route to the references below. These references own the current signatures and outcomes.
+
+Important: allow.user conditions check the logged-in user's session data — not data source records. Always use literal values here (e.g., "Admin", "Manager"). Although Handlebars syntax is supported, using it in allow.user (e.g., {% raw %}{{ user.[Department] }}{% endraw %}) compares the user's field against itself — always true, granting access to any logged-in user. Handlebars templates are useful in require conditions, where they scope queries by the user's data. See Data requirements .
-
-{% raw %}
-```json
-{
- "allow": {
- "user": {
- "Role": { "equals": "Manager" },
- "Status": { "notequals": "Inactive" }
- }
- }
-}
-```
-{% endraw %}
-
-**Specific API token:**
-
-```json
-{
- "allow": {
- "tokens": [42857]
- }
-}
-```
-
-### Restricting columns
-
-For reads, use `include` to allow specific returned columns or `exclude` to hide columns. If both are specified, `include` takes precedence. Without either, all columns are returned. For individual inserts and updates, `exclude` removes named fields before saving; `include` does not restrict writes. An exact write allowlist requires independent server enforcement; see [Data requirements](#data-requirements-and-query-validation).
-
-```json
-{
- "type": ["select"],
- "allow": "loggedIn",
- "exclude": ["Password", "SSN", "Salary"]
-}
-```
-
-Security note: Always exclude privilege fields (e.g., Admin, Role, Permissions) from rules that allow logged-in users to insert or update records. Otherwise, users can escalate their own privileges.
-
-### Example: role-based access with protected fields
-
-This example shows an "Employees" data source where admins have full access, regular users can read records (without sensitive fields), and can only update their own record or insert records with a non-privileged role.
-
-**Data source columns:**
-
-| Email | First Name | Role | Department | Salary | Password | Admin | Permissions |
-|---|---|---|---|---|---|---|---|
-| alice@acme.com | Alice | Admin | Engineering | 150000 | ••••• | Yes | all |
-| bob@acme.com | Bob | User | Marketing | 85000 | ••••• | No | read |
-| carol@acme.com | Carol | User | Engineering | 95000 | ••••• | No | read |
-
-**Security rules:**
-
-{% raw %}
-```json
-[
- {
- "type": ["select", "insert", "update", "delete"],
- "allow": { "user": { "Role": { "equals": "Admin" } } },
- "enabled": true
- },
- {
- "type": ["select"],
- "allow": "loggedIn",
- "exclude": ["Password", "Salary"],
- "enabled": true
- },
- {
- "type": ["update"],
- "allow": "loggedIn",
- "require": [
- { "Email": { "equals": "{{user.[Email]}}" } }
- ],
- "exclude": ["Role", "Admin", "Permissions"],
- "enabled": true
- },
- {
- "type": ["insert"],
- "allow": "loggedIn",
- "require": [
- "Email",
- "First Name",
- { "Email": { "equals": "{{user.[Email]}}" } },
- { "Role": { "equals": "User" } }
- ],
- "exclude": ["Admin", "Permissions"],
- "enabled": true
- }
-]
-```
-{% endraw %}
-
-**Queries that succeed:**
-
-When Alice (Admin) is logged in, the first rule matches — she has full access to all records and columns:
-
-```js
-// JS API — Alice reads all records including sensitive fields
-const connection = await Fliplet.DataSources.connectByName('Employees');
-const allRecords = await connection.find();
-// Returns all 3 records with all columns including Salary and Password
-```
-
-```plaintext
-// REST API equivalent
-POST /v1/data-sources/123/data/query
-Auth-token:
-Content-Type: application/json
-
-// "type" here is the REST API operation type, not a security rule field
-{ "type": "select" }
-
-// Response (200 OK): all entries with all columns
-```
-
-When Bob (User) is logged in, the second rule matches for reads — he can see records but `Password` and `Salary` are excluded:
-
-```js
-// JS API — Bob reads records (sensitive columns automatically excluded)
-const connection = await Fliplet.DataSources.connectByName('Employees');
-const records = await connection.find();
-// Returns all 3 records, but each record is missing Password and Salary
-// e.g. { Email: "alice@acme.com", "First Name": "Alice", Role: "Admin", Department: "Engineering", Admin: "Yes", Permissions: "all" }
-```
-
-Bob can update his own record (the third rule checks the stored record's email):
-
-```js
-// JS API — Bob updates his own name
-const connection = await Fliplet.DataSources.connectByName('Employees');
-await connection.update(456, {
- Email: 'bob@acme.com',
- 'First Name': 'Robert'
-});
-// Succeeds: the stored Email matches Bob's session; protected fields are excluded
-```
-
-```plaintext
-// REST API equivalent
-PUT /v1/data-sources/123/data/456
-Auth-token:
-Content-Type: application/json
-
-{ "Email": "bob@acme.com", "First Name": "Robert" }
-
-// Response (200 OK): updated entry
-```
-
-**Queries that fail:**
-
-When a query is denied, the API responds with HTTP status **400** and a JSON error body. The `message` field changes based on the denied operation — for example, `"do not allow this app to read data"` for selects, or `"do not allow this app to insert data"` for inserts:
-
-```json
-{
- "message": "The security rules for the Data Source \"Employees\" do not allow this app to read data.",
- "type": "datasource.access",
- "payload": { "dataSourceId": 123 }
-}
-```
-
-In the JS API, the promise is rejected with this error object. The `message` adapts to include the data source name and the specific operation that was denied (read, insert, update, or delete). The `type` is always `"datasource.access"` and `payload.dataSourceId` identifies the data source.
-
-```js
-// Bob tries to update another user's record — fails
-// (The stored Email does not match Bob's session, regardless of the submitted Email)
-await connection.update(789, {
- Email: 'carol@acme.com',
- 'First Name': 'Carolina'
-});
-
-// Bob tries to escalate his own role — fails
-// (Role is excluded from the update rule, so it cannot be written)
-await connection.update(456, {
- Email: 'bob@acme.com',
- Role: 'Admin'
-});
-
-// Bob tries to insert a record with an admin role — fails
-// (the insert rule requires Role: "User")
-await connection.insert({
- Email: 'bob@acme.com',
- 'First Name': 'Bob Clone',
- Role: 'Admin'
-});
-
-// Bob tries to delete a record — fails
-// (no delete rule matches for non-admin users)
-await connection.removeById(789);
-```
-
-### Example: department-scoped access
-
-In this example, managers see all records for their department, regular users only see their own record, and inserts are restricted to the user's department. Department scoping is enforced through `require` conditions on the query, not through `allow.user` (which checks the user's identity, not the data being accessed).
-
-**Data source columns:**
-
-| Email | Name | Role | Department | Salary | ManagerNotes |
-|---|---|---|---|---|---|
-| alice@acme.com | Alice | Manager | Engineering | 150000 | Top performer |
-| bob@acme.com | Bob | User | Engineering | 85000 | |
-| carol@acme.com | Carol | User | Marketing | 95000 | |
-| dave@acme.com | Dave | Manager | Marketing | 140000 | Needs training |
-
-**Security rules:**
-
-{% raw %}
```json
[
- {
- "type": ["select"],
- "allow": {
- "user": { "Role": { "equals": "Manager" } }
- },
- "require": [
- { "Department": { "equals": "{{user.[Department]}}" } }
- ],
- "exclude": ["Salary"],
- "enabled": true
- },
- {
- "type": ["select", "update"],
- "allow": "loggedIn",
- "require": [
- { "Email": { "equals": "{{user.[Email]}}" } }
- ],
- "exclude": ["Salary", "ManagerNotes"],
- "enabled": true
- },
- {
- "type": ["insert"],
- "allow": "loggedIn",
- "require": [
- "Name",
- { "Department": { "equals": "{{user.[Department]}}" } },
- { "CreatedBy": { "equals": "{{user.[Email]}}" } }
- ],
- "exclude": ["Role", "Admin"],
- "enabled": true
- }
+ { "type": ["select"], "allow": "all" }
]
```
-{% endraw %}
-
-The first rule uses `allow.user` to check the user's role (a literal match — "is this user a Manager?") and `require` to enforce department scoping on the query (the `where` clause must include the user's own department). This combination ensures only managers can use the rule *and* they can only query their own department's data.
-
-**Queries that succeed:**
-
-When Alice (Manager, Engineering) is logged in, the first rule matches — she is a Manager, and her query includes `Department: 'Engineering'` which satisfies the `require` condition:
-
-```js
-// JS API — Alice reads all Engineering records
-const connection = await Fliplet.DataSources.connectByName('Staff');
-const records = await connection.find({
- where: { Department: 'Engineering' }
-});
-// Returns Alice and Bob's records, excluding Salary
-// e.g. { Email: "bob@acme.com", Name: "Bob", Role: "User", Department: "Engineering", ManagerNotes: "" }
-```
-
-```plaintext
-// REST API equivalent
-POST /v1/data-sources/123/data/query
-Auth-token:
-Content-Type: application/json
-
-{ "type": "select", "where": { "Department": "Engineering" } }
-
-// Response (200 OK): Alice and Bob's entries, Salary excluded
-```
-
-When Bob (User, Engineering) is logged in, the first rule does not match (wrong role). The second rule matches — he must filter by his own email:
-
-```js
-// JS API — Bob reads his own record
-const connection = await Fliplet.DataSources.connectByName('Staff');
-const records = await connection.find({
- where: { Email: 'bob@acme.com' }
-});
-// Returns Bob's record only, excluding Salary and ManagerNotes
-```
-
-Bob can insert a record in his own department:
-
-```js
-// JS API — Bob creates a record in Engineering
-await connection.insert({
- Name: 'New Hire',
- Department: 'Engineering',
- CreatedBy: 'bob@acme.com'
-});
-// Succeeds — all require conditions are met, Role and Admin are excluded from writes
-```
-```plaintext
-// REST API equivalent (Fliplet uses PUT for both inserts and updates on this endpoint)
-PUT /v1/data-sources/123/data
-Auth-token:
-Content-Type: application/json
+Use this only for data intended for anonymous readers. It grants no insert, update or delete operation. The [public lookup example](API/datasources/security-examples.md#public-lookup) pairs this rule with sample rows and SDK calls.
-{ "Name": "New Hire", "Department": "Engineering", "CreatedBy": "bob@acme.com" }
+## Choose the right guide
-// Response (200 OK): new entry
-```
-
-**Queries that fail:**
-
-```js
-// Alice (Manager) tries to read Marketing records — fails
-// (the require condition demands Department matching Alice's own department "Engineering";
-// "Marketing" does not match, so the first rule is skipped; no other rule grants broad access)
-await connection.find({
- where: { Department: 'Marketing' }
-});
-
-// Bob tries to read all records without filtering by email — fails
-// (the second rule requires Email matching Bob's session; without a where clause,
-// the require condition is not met, the rule is skipped, and no subsequent rule grants access)
-await connection.find();
-
-// Bob tries to read Alice's record — fails
-// (Email does not match Bob's session)
-await connection.find({
- where: { Email: 'alice@acme.com' }
-});
-
-// Bob tries to insert a record in Marketing — fails
-// (Department does not match Bob's department "Engineering")
-await connection.insert({
- Name: 'Spy',
- Department: 'Marketing',
- CreatedBy: 'bob@acme.com'
-});
-
-// Bob tries to insert with a different CreatedBy — fails
-// (CreatedBy must equal Bob's email)
-await connection.insert({
- Name: 'Fake',
- Department: 'Engineering',
- CreatedBy: 'alice@acme.com'
-});
-```
-
-## Data requirements and query validation
-
-The `require` property defines conditions that incoming queries must satisfy. This is not the same as querying data — it controls how queries are **assessed against the rule's data requirements**.
-
-Components such as List from Data Source , Chart and Form may issue broad queries that do not satisfy your read requirements. Configure their filters or use custom queries that satisfy the intended restrictions. Adding a permissive fallback rule grants broader access even when a stricter rule appears first; an appId restriction scopes a rule to apps, not to individual components.
-
-The behavior of `require` varies by operation type:
-
-- **For `select` queries:** Requirements validate the submitted filter. For example, if a rule requires {% raw %}`{ "Email": { "equals": "{{user.[Email]}}" } }`{% endraw %}, a `find()` call must include `where: { Email: user.Email }` (or an equivalent supported filter) for that rule to grant access. Direct reads by entry ID instead validate the stored entry data. To obtain session data client-side, use the [Session API](API/fliplet-session.html).
-- **For `delete` operations:** Direct deletion by entry ID validates the stored entry data and does not require a client `where` filter. Query-based deletion requires a nonempty filter and read access, then validates delete access against each matched row's stored data. Explicit commit deletion IDs provide an array of existing row data; other bulk deletion paths, such as replacing imported data, can omit row data. Do not assume every delete evaluates a submitted filter or always provides stored rows.
-- **For `insert` operations:** Required fields and value conditions validate the submitted data.
-- **For `update` operations:** String requirements check that the submitted data includes the named fields. Object-form value conditions check the existing stored row, including stored rows loaded for commit updates. They do not restrict proposed replacement values. The current custom-rule `query` context also cannot reliably enforce allowed new values or an exact set of write fields. Use a server write mechanism that independently enforces such constraints, or leave the constrained write unavailable.
+| Task | Guide |
+|---|---|
+| Understand rule order, identity, requirements and endpoint differences | [Security rule reference](API/datasources/security-rules.md) |
+| Explore public, shared, owner-filtered reads, profile or role/state examples | [Security examples](API/datasources/security-examples.md) |
+| Prove allowed and denied app behavior | [Authenticated preview testing](API/datasources/testing-security.md) |
+| Connect, query or write through JavaScript | [Fliplet.DataSources reference](API/fliplet-datasources.md) |
+| Configure authentication or hooks | [App authentication](API/v3/auth.md), [Session API](API/fliplet-session.md) and [Data source hooks](Data-Source-Hooks.md) |
-Rules grant access independently: If any require condition fails, that rule does not grant access and evaluation continues for reads and writes alike. Access is denied only if no applicable rule grants it. Remove or narrow any other rule that would admit requests your restrictions are intended to reject; ordering a strict rule first does not make a broader fallback safe.
+## Check the policy's limits
-### Data requirement types
+Rules grant access in order. A failed rule falls through for writes as well as reads, so a later broad grant can defeat an earlier restriction.
-`require` accepts an array of strings (required field names) and objects (field conditions). Four requirement types are available:
+Object conditions on updates check the **stored row**, while string requirements check submitted key presence. A condition on stored `Owner` or `Status` decides whether the caller may update that row; it does not stop them changing those fields. Exclude the fields each actor must not set from that actor's insert and update rules: single-record writes and `commit()` both drop excluded fields, and commit updates merge onto the stored row. Bulk `append()` and `replaceWith()` do not apply exclusions. Custom scripts receive a copied input; assigning audit fields to that copy does not reliably persist them.
-| Requirement type | Description | Query patterns that satisfy this requirement |
-|---|---|---|
-| Field is required | The query must include the specified column | Any query that provides the column as a key |
-| Field equals | The query value for the column must match exactly | `{ Field: value }` or `{ Field: { $eq: value } }` |
-| Field not equals | The query value for the column must not match the specified value | `{ Field: { $ne: value } }` or any query where the field's value differs from the specified value |
-| Field contains | The query value for the column must contain the specified substring | `{ Field: value }` or `{ Field: { $iLike: value } }` or `{ Field: { $like: value } }` |
+For workflows where one person submits and another decides, see [requests and approvals](API/datasources/security-examples.md#requests-and-approvals).
-These requirement types (equals, notequals, contains) are not query operators — they define how the security rule validates incoming queries. For the full list of query operators you can use when reading and writing data, see the query operators reference .
+## Existing section links
-### Handlebars templating
+These destinations retain links to the previous sections of this guide.
-Condition values can reference the logged-in user's session data using Handlebars syntax. This enables dynamic, per-user access scoping. Use bracket notation (e.g., `[Email]`) for field names — it is required for names with spaces or special characters (e.g., {% raw %}`{{user.[First Name]}}`{% endraw %}), and also works for simple names. Plain dot notation ({% raw %}`{{user.Email}}`{% endraw %}) works only for field names without spaces.
+Important: When a rule has a script, the script is the sole determinant of access. Standard rule fields like allow and type are ignored — the script runs regardless of login status or operation type. Your script must perform its own identity and operation checks (e.g., if (!user) return { granted: false };). If the script does not return a value (e.g., an unhandled operation type falls through without a return), access is denied by default .
+[Data requirements and query validation](API/datasources/security-rules.md#requirements-by-operation)
-When writing a custom rule, these variables are available in the script context:
+Important: You must return an object — bare boolean values (e.g., return true) are not supported and will be treated as a denial. Always use return { granted: true } or return { granted: false }.
+[Granting access](API/datasources/security-rules.md#custom-scripts)
Cross-data-source lookups add a database round-trip per request and are not cached. Keep queries efficient and use findOne when you only need a single record. Script execution has a 3-second timeout — if the script does not return within 3 seconds, access is denied. Async operations (like DataSources queries) are supported via await.
-
-Connect using the data source ID (number) or name (string):
-
-```js
-if (!user) {
- return { granted: false };
-}
-
-if (type === 'select') {
- const entry = await DataSources(123).findOne({
- where: {
- Office: user.Office,
- Managers: { $in: user.Manager },
- Country: { $like: '%United Kingdom%' }
- }
- });
-
- // Allow reading data if the user has a manager in the same office
- if (entry) {
- return { granted: true };
- }
-}
-
-if (type === 'insert') {
- const entries = await DataSources('Users').find();
-
- // Only allow writes as long as there are fewer than 10 entries in the target Data Source
- if (entries && entries.length < 10) {
- return { granted: true };
- }
-}
-```
+[Validating submitted filters](API/datasources/security-rules.md#script-inputs-and-mutation)
-`find` returns an array of matching entries (empty array if none match). `findOne` returns a single entry object, or `null` if no match is found.
+Use DataSources('Name') (data source name) instead of DataSources(123) (ID) in custom scripts. Data source names are preserved during app clone, so name-based lookups continue to work without manual remapping.
\ No newline at end of file
+[Reading data from other data sources](API/datasources/security-rules.md#reading-another-data-source)
diff --git a/docs/bin/__tests__/datasource-reference-docs.test.mjs b/docs/bin/__tests__/datasource-reference-docs.test.mjs
new file mode 100644
index 00000000..e8f4a941
--- /dev/null
+++ b/docs/bin/__tests__/datasource-reference-docs.test.mjs
@@ -0,0 +1,47 @@
+import { spawnSync } from 'node:child_process';
+import { it } from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync, mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
+import { join, dirname } from 'node:path';
+import { tmpdir } from 'node:os';
+import { fileURLToPath } from 'node:url';
+import { collectDocs, validateFrontmatter, emitV3LibraryCatalog, CLUSTERS } from '../build-agent-indexes.mjs';
+import { copySiblings } from '../copy-md-siblings.mjs';
+const source = fileURLToPath(new URL('../../', import.meta.url));
+const paths = ['API/fliplet-datasources.md', ...['reading-data', 'writing-data', 'file-attachments', 'subscriptions', 'offline-database', 'managing-data-sources', 'query-operators', 'joins', 'views'].map(name => `API/datasources/${name}.md`)];
+it('discovers split SDK references and copies exact raw pages without adding namespace owners', () => {
+ const root = mkdtempSync(join(tmpdir(), 'datasource-reference-'));
+ try {
+ for (const path of paths) {
+ mkdirSync(dirname(join(root, path)), { recursive: true });
+ writeFileSync(join(root, path), readFileSync(join(source, path)));
+ }
+ const docs = collectDocs(root);
+ assert.equal(docs.length, paths.length);
+ assert.deepEqual(validateFrontmatter(docs), []);
+ const cluster = CLUSTERS.find(cluster => cluster.name === 'fliplet-data-sources');
+ for (const path of paths) assert.ok(cluster.match(path), path);
+ const catalog = emitV3LibraryCatalog(docs);
+ assert.equal(catalog.libraries.length, 1);
+ assert.equal(catalog.libraries[0].docUrl, 'https://developers.fliplet.com/API/fliplet-datasources.md');
+ const out = join(root, '_site'); mkdirSync(out);
+ assert.equal(copySiblings(root, out), paths.length);
+ for (const path of paths) assert.equal(readFileSync(join(out, path), 'utf8'), readFileSync(join(source, path), 'utf8'));
+ } finally { rmSync(root, { recursive: true, force: true }); }
+});
+it('SDK reference JSON and plain JavaScript examples parse independently', () => {
+ const AsyncFunction = Object.getPrototypeOf(async function() {}).constructor;
+ let blocks = 0;
+ for (const path of paths) {
+ const body = readFileSync(join(source, path), 'utf8');
+ for (const match of body.matchAll(/```(json|js)\n([\s\S]*?)```/g)) {
+ blocks++;
+ if (match[1] === 'json') JSON.parse(match[2]);
+ else if (/^import /m.test(match[2])) {
+ const parsed = spawnSync(process.execPath, ['--check', '--input-type=module'], { input: match[2], encoding: 'utf8' });
+ assert.equal(parsed.status, 0, `${path}: ${parsed.stderr}`);
+ } else assert.doesNotThrow(() => new AsyncFunction(match[2]), path);
+ }
+ }
+ assert.ok(blocks >= 20);
+});
diff --git a/docs/bin/__tests__/datasource-security-docs.test.mjs b/docs/bin/__tests__/datasource-security-docs.test.mjs
new file mode 100644
index 00000000..5b739824
--- /dev/null
+++ b/docs/bin/__tests__/datasource-security-docs.test.mjs
@@ -0,0 +1,110 @@
+import { it } from 'node:test';
+import assert from 'node:assert/strict';
+import { mkdtempSync, mkdirSync, readFileSync, writeFileSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join, dirname } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { CLUSTERS, collectDocs, emitLlmsTxt, emitV3LibraryCatalog, validateFrontmatter } from '../build-agent-indexes.mjs';
+import { copySiblings } from '../copy-md-siblings.mjs';
+
+const source = fileURLToPath(new URL('../../', import.meta.url));
+const paths = ['Data-source-security.md', 'API/fliplet-datasources.md',
+ 'API/datasources/security-rules.md', 'API/datasources/security-examples.md',
+ 'API/datasources/testing-security.md'];
+const moved = ['security-rules', 'access-rule-structure', 'defining-who-can-access',
+ 'restricting-columns', 'example-role-based-access-with-protected-fields',
+ 'example-department-scoped-access', 'data-requirements-and-query-validation',
+ 'data-requirement-types', 'handlebars-templating', 'require-syntax',
+ 'custom-security-rules', 'granting-access', 'modifying-the-input-query',
+ 'checking-data-when-committing-changes', 'reading-data-from-other-data-sources'];
+
+it('discovers every security page in the existing cluster and retains one SDK catalog owner', () => {
+ const root = mkdtempSync(join(tmpdir(), 'datasource-security-docs-'));
+ try {
+ for (const path of paths) {
+ mkdirSync(dirname(join(root, path)), { recursive: true });
+ writeFileSync(join(root, path), readFileSync(join(source, path)));
+ }
+ const docs = collectDocs(root);
+ assert.equal(docs.length, paths.length);
+ assert.deepEqual(validateFrontmatter(docs), []);
+ const cluster = CLUSTERS.find(item => item.name === 'fliplet-data-sources');
+ const index = emitLlmsTxt(docs);
+ for (const path of paths) {
+ assert.ok(cluster.match(path), path);
+ assert.ok(index.includes(`https://developers.fliplet.com/${path}`), path);
+ }
+ const catalog = emitV3LibraryCatalog(docs);
+ assert.equal(catalog.libraries.length, 1);
+ assert.equal(catalog.libraries[0].docUrl, 'https://developers.fliplet.com/API/fliplet-datasources.md');
+ const site = join(root, '_site');
+ mkdirSync(site);
+ assert.equal(copySiblings(root, site), paths.length);
+ for (const path of paths) assert.equal(readFileSync(join(site, path), 'utf8'), readFileSync(join(source, path), 'utf8'));
+ } finally {
+ rmSync(root, { recursive: true, force: true });
+ }
+});
+
+it('retains every moved public security fragment with an explicit destination link', () => {
+ const body = readFileSync(join(source, paths[0]), 'utf8');
+ for (const anchor of moved) {
+ assert.match(body, new RegExp(String.raw`'), examples.indexOf('## Editable profiles'));
+ const policy = JSON.parse(section.match(/```json\n(\[\s*\{\s*"type"[\s\S]*?)```/)[1]);
+ assert.deepEqual(policy[0].type, ['select']);
+ assert.ok(section.includes('`replaceWith()` access still need their own tests'));
+});
diff --git a/docs/v3/capabilities.md b/docs/v3/capabilities.md
index 83fd3fa3..ac304a1f 100644
--- a/docs/v3/capabilities.md
+++ b/docs/v3/capabilities.md
@@ -16,7 +16,7 @@ Every Fliplet JS API available to V3 apps, grouped by capability category. Each
## Data
- [`Fliplet.Cache`](https://developers.fliplet.com/API/core/cache.html) **(preloaded)** — Run async operations once and memoize their results, with optional expiry and background refresh.
-- [`Fliplet.DataSources`](https://developers.fliplet.com/API/fliplet-datasources.html) — Connect to, query, insert, update, and delete records in Fliplet Data Sources from inside an app. All methods are promise-based.
+- [`Fliplet.DataSources`](https://developers.fliplet.com/API/fliplet-datasources.html) — Connect to Fliplet data sources and find references for queries, writes, subscriptions, native storage and source management.
- [`Fliplet.DataSources.Encryption`](https://developers.fliplet.com/API/fliplet-encryption.html) — Automatically encrypt and decrypt selected Data Source columns on-device by registering a private key and column list.
- [`Fliplet.Storage and Fliplet.App.Storage`](https://developers.fliplet.com/API/core/storage.html) **(preloaded)** — Persist JSON-serializable values to device or browser storage, scoped globally or to the current app.
@@ -31,7 +31,7 @@ Every Fliplet JS API available to V3 apps, grouped by capability category. Each
## Communications
-- [`Fliplet.Chat`](https://developers.fliplet.com/API/fliplet-chat.html) — Build one-to-one, group, and public-channel chat features. Fliplet.Chat owns the conversations and messages data sources internally — supply only the contacts list (who can chat with whom).
+- [`Fliplet.Chat`](https://developers.fliplet.com/API/fliplet-chat.html) — Chat API for V2 apps with a Chat component. Not available to V3 apps: build messaging on the app's own data sources.
- [`Fliplet.Communicate`](https://developers.fliplet.com/API/fliplet-communicate.html) — Send email, SMS, push notifications, and share URLs from a Fliplet app using a single Communicate namespace.
- [`Fliplet.Navigator.Notifications`](https://developers.fliplet.com/API/core/notifications.html) **(preloaded)** — Check notification support, request permission, and send local device notifications from JavaScript.
- [`Fliplet.Notifications`](https://developers.fliplet.com/API/fliplet-notifications.html) — Read, send, and schedule in-app and push notifications in Fliplet apps, with support for scopes, read receipts, and badge counts.